Extreme Probabilities Project

Manifold Community Fund Forecasting

🐶

Evan Daniel

CompleteImpact certificate

$0raised

$3,000valuation

Project summary

Currently, prediction markets have trouble pricing extreme probabilities. Roughly speaking, getting more confident than about 95% is very difficult, and distortion appears even by 90% on relevant time frames. Nonetheless, many markets of public interest trade in this regime -- nuclear war and other catastrophes being a prime example.

I'm attempting to create a template for derivative markets, trading bots, and subsidy structures that will improve on this. The project is in progress already. The full plan is not yet written up, but hopefully the current work is enough that one can see where it's going: https://manifold.markets/browse?topic=extreme-probabilities-project

Roughly speaking, the template is this: create tranches of related or unrelated high-confidence markets, and a derivative on how many resolve in the expected direction. Traders can bet on the group, mildly reducing information costs and trading friction and improving their potential return. Bots can arb against the underlying. And traders with specific knowledge can touch up the low-probability events mostly by betting on the individual low-prob events to happen. For example, betting Yes on "10" here lets you play the house (in conjunction with the bots) against folks who want good odds on weird stuff: https://manifold.markets/EvanDaniel/long-shot-bets-how-many-of-these-10

The project is to continue along that path. It will include a mix of further writeups of the plan, creating and subsidizing markets to support it, and bounties for trading bots and feature additions to Manifold.

What are this project's goals and how will you achieve them?

Minimally successful: better quantify the effects that cause this. Be able to say things like "the market believes these markets are mispriced" in a useful way. Improve our understanding of extreme-probability pricing behavior, interest rate effects, and so on. This goal is well underway; it already looks like we have minimal data to support it, but it has not been analyzed in detail or written up.

Moderate success: rich enough data to chart a path forward. Currently I can guess where some stumbling blocks are; chart them in detail. Produce a set of feature requests that Manifold or another prediction market could implement to improve the situation, with enough data and analysis for the case to be convincing.

Extreme success: those feature requests get implemented. They do in fact work as desired. There exists a template for market structure and subsidy structure required to produce a significant improvement in market accuracy at extreme probabilities, at least on markets with enough interest and public value. Manifold charts new territory, solving a problem that has plagued prediction markets for decades.

How will this funding be used?

A mix of labor and cash costs. Labor is split between my time and bounties. Bounties will be to pay for software development work, including arb bots and hopefully also feature additions to the Manifold code base. My time will include market creation, analysis, and blog post writing. Non-bounty cash costs will include market creation and subsidization.

This project does not neatly fit in any defined category for the Manifold Community Fund. There are several other promising submissions so far. The fund is small, at only $30k, and capturing even 10% of it seems optimistic.

I'm pricing this on the assumption that investors are unlikely to make a profit off it. I don't see a way to make investors a profit off only the community fund grants, while also paying anything resembling fair prices for either my time or the software development work I plan to post bounties on.

A guess at a budget (not a commitment in any way):

Market creation, subsidies, and standing limit orders: $200
Bounties for bots and features: $1500+
Bot investment / capitalization: $200+
My labor: $1000+

The "extreme success" version of this certainly requires more budget for labor hours priced at a fair wage. The minimum valuation reflects this. If things are looking promising I plan to sell my equity and/or fund the above budget out of pocket; I'm in a position to afford that, and the minimum funding level is adequate to ensure that the above budget or equivalent is available.

Who is on your team and what's your track record on similar projects?

Just me. Hopefully also bounty respondents.

I don't have meaningful experience on a project like this. However, my professional background includes technical writing, math, programming, and other engineering work. I've been participating in prediction markets for over 20 years.

What are the most likely causes and outcomes if this project fails? (premortem)

Possibly the problem is just too hard and my plan is in fact not adequate to solve it or even make meaningful headway.

This will not work without interest and support from the broader community. If I (or the community) cannot identify interesting markets, and generate liquidity on them, we won't get accurate pricing info to inform the project.

If things aren't seeing public interest and support, I'll probably give up on it somewhere midway to the "moderate success" state.

I don't have the time to do the software work myself; my software dev skills are quite rusty and merely adjacent to what's needed. If I don't attract any interest and/or don't have enough funding to pay for development bounties the project will likely fizzle somewhere between minimal and moderate success.

What other funding are you or your project getting?

None. I'm supplying some out of pocket cash so far, and will supply a bit more even if this gets no funding.

Manifold Markets

10 months ago

January 15th Evaluation Results

Valuations: 0, 0, 0, 0, 0, 0, 0, 0

Weighted Median Valuation: 0

Eliza: Zero because I don't think he made much progress on it, but I still think that it is critically important that Manifold figure out a way to correctly price extreme probabilities. This is one of the most unintuitive parts about forecasting and giving 'regular people' the ability to understand the difference between 1 in 100 and 1 in 1000 and 1 in a million seems like one of the most +EV things Manifold could do. These are the kind of probabilities that affect life-altering risks. I hope someone can find a way to support or work with Evan on this.

cc6: Nothing done yet

Isaac King: An extremely important problem to solve, but this project hasn't solved it yet, or made any progress as far as I can tell. $0.

wasabipesto: It doesn't look like there is any activity on this project.

Rachel: No progress yet.

David Chee: Doesn't seem to be any work produced yet.

SG: No comment

Austin: No updates yet? Excited to see what Evan’s been up to though!

🥭

Sylvie Liberman

10 months ago

would love to fund this

Nuño Sempere

11 months ago

I thought bundling was a neat idea. Contra other comments I don't think this would only be valuable if you also solved discounting. And discounting could maybe be achieved by the platform offering to match your returns (on resolved or exited markets).

Manifold Markets

11 months ago

December 15th Evaluation Results

Valuations: 0, 0, 0, 0, 50, 200

Median Valuation: 0

David Chee: Think it's unlikely it would work, but would be extremely valuable if it did. So I don't think $5000 is a bad asking price at all. But no work produced yet.

SG: It seems they haven't released anything yet. Not sure they're on track to solve the problem of exteme probabilities.

Rachel: I think this is pretty important, and unlike most other projects, makes the experience of Manifold better for everyone, not just power users. Plus it makes the information Manifold can provide to the public more helpful, and might help other prediction market platforms too. Big fan of the goal, as it seems like other Manifold mods are! But no reported progress so far, so have to give a valuation of 0.

Inga Wei: I think this may depend too much on curation and community to be wildly successful

Ian Philips: Your markets look interesting and I'm happy your thinking about how to improve Manifold! The only problem at this point in time is I don't think this is super relevant to Manifold's success.

Austin: This is a pretty ambitious goal, and one that would benefit the Manifold community (and the world!) a lot — if you can discover a way to do so. As of right now, there’s a few markets created with interesting ideas behind them. I think these templates are somewhat valuable (one thing I’d be happy to fund is something like “a directory of easily duplicatable market templates for a variety of common scenarios, eg date, numeric, low% markets”). However, my current valuation reflects the fact that these templates have not yet been adopted widely.

Isaac King

12 months ago

I've created a rate of return calculator that shows your rate of return on a market in monthly-compounding-interest-equivalent. Manifold's average growth in mana is about 4% a month, so any market that offers <4% a month is irrational to bet in no matter how highly subsidized it is or how low you get the transaction costs.

https://outsidetheasylum.blog/long-term-bet-calculator

Admittedly, there are a lot of existing markets that offer >4% and have still not been bet on sufficiently, so I think I stand corrected on the whole. Trading bots and subsidies can potentially be of help on some markets in 1% - 10%, 1 month - 5 year range.

Tetra Jones

12 months ago

I introduced my own insane scheme to finally get good calibration on very unlikely questions, which I don't think worked all that well though it's hard to tell given that it predictably resolved N/A https://manifold.markets/Tetraspace/100-amplified-odds-details-within-w

Would be very interesting to see how this scheme goes!

🐶

Eliza

12 months ago

I don't even care if it is hard, any work toward this is positive for Manifold.

I do not want to look at Manifold and see a 99.9% event displayed as 98%. It bothers me.

Isaac King

about 1 year ago

I'm a big fan of this goal, and I would be willing to fund it conditional upon success, but I unfortunately expect it to fail.

🐶

Evan Daniel

about 1 year ago

@Isaac-King I would be interested in hearing serious offers for shares conditional on "moderate success" or better, as defined above. I'm very uncertain about the "extreme success" outcome. At the very least, it isn't going to be achieved without far more serious dev support from Manifold than I have any reason to count on.

Isaac King

about 1 year ago

@evand I'm not really interested in the "moderate success" goal; coming up with plans is easy, it's putting them into practice that's the challenge. Conditional on "extreme success" (using my judgement on what level of improvement is enough to count) I'd be happy to fund the full $750.

The reason I expect this to fail is because your proposal doesn't include anything to address the core issue of discounting rates. Adding subsidies only decreases the relative temporal transaction costs of betting, asymptotically towards 0. But even with no transaction costs whatsoever it's still irrational to bet in a market if a higher rate of return can be found elsewhere, which is almost always true for a market at <3% that resolves in more than a month.

Arbitrage bots and derivative markets are just further epicycles that don't address the core issue either. A real solution will need to fundamentally change the market structure such that the rate of return is competitive. The two best suggestions I've seen thus far are some sort of log payout market structure developed by Robin Hanson, and an increased loan rate. (The latter runs into massive bankruptcy risk.)

🐶

Evan Daniel

about 1 year ago

@Isaac-King That seems entirely fair! Though, to be clear, the "moderate success" case is not just coming up with a plan; it also includes supporting that plan with data. Hopefully the data does in fact make things more interesting! "I'll just write up my plan" is a thing I could do, right now, at a reasonable cost of a few more hours of work, and justifies far less than $5k.

If we actually hit the "extreme success" case, this is worth far more than $5k. This is a problem that has existed for decades without serious answers. AFAICT, the best suggestion so far is something like "bet in denomination of stock index funds" or "joint markets of the thing of interest x stock prices", which are both good and correct ideas but not actually all that exciting and only address a piece of the problem.

Per your market, I don't actually expect I'll fix the problem in the 99%+ range. I'd be very happy with a substantial improvement in the 95%-98.5% range, especially if combined with evidence that the same plan could work at more extreme probabilities, at least in a subset of interesting cases.