Repository navigation
Conversation
blishko
left a comment
There was a problem hiding this comment.
I still have to look at the program generator, but this looks interesting!
blishko
left a comment
There was a problem hiding this comment.
I am slowly getting through this PR.
I have looked at YulProgramGenerator and did not notice anything suspicious.
I wish the code would be simpler, but not sure if that is possible.
In any case, it seems to me a few things could be separated out of this PR, like the extra yulOptimizerTests and the missing headers.
Keeping the PR focused is better for reviewing.
| libyul/DataFlowAnalyzerPropertyTest.cpp | ||
| libyul/DominatorTreePropertyTest.cpp | ||
| libyul/OptimizerStepSequence.cpp | ||
| libyul/OptimizerStepSequence.h | ||
| libyul/StackLimitEvaderPropertyTest.cpp | ||
| libyul/StackShufflerPropertyTest.cpp | ||
| libyul/TarjanSCCPropertyTest.cpp | ||
| libyul/TLSFFreeListPropertyTest.cpp | ||
| libyul/YulProgramGenerator.cpp | ||
| libyul/YulProgramGenerator.h |
There was a problem hiding this comment.
I think we need to organize these tests a bit more. Not necessarily here, more likely as a follow-up.
| ); | ||
| static_assert( | ||
| smallValueBound > 1, | ||
| "The calldata of all ones must be made of small values, otherwise it is not what the programs expect to read." |
There was a problem hiding this comment.
I can't quite parse this statement. "calldata of all ones" What does that mean?
There was a problem hiding this comment.
It was about the calldatawords that are fixed 0,0,0,0 or 1,1,1,1 or rnd(0,smallValueBound-1),rnd(0,smallValueBound-1),rnd(0,smallValueBound-1),rnd(0,smallValueBound-1).
It's now less obscure I hope...
|
|
||
| /// @returns a number below @a _count. | ||
| /// Draw into named variables: the evaluation order of function arguments is unspecified. | ||
| std::size_t pick(std::size_t const _count) |
There was a problem hiding this comment.
| std::size_t pick(std::size_t const _count) | |
| std::size_t pick(std::uint8_t const _count) |
maybe?
There was a problem hiding this comment.
I don't understand what we'd gain with this? I see some potential issues, but I don't see the gain. Maybe I'm blind. Can you explain?
There was a problem hiding this comment.
I don't remember anymore why I thought it would be a good idea.
Please ignore.
There was a problem hiding this comment.
I think you had the right idea. This is only meaningful for _count <= 256 if the input is a single byte. I'd add an assert instead of changing the signature though:
yulAssert(0 < _count && _count <= 256, "pick draws from a single byte");There was a problem hiding this comment.
Yes, fair, and the return type can be uint8_t. Fixing.
79b1fc5 to
a26a03f
Compare
Thanks to @blishko for ithe idea
…ting on it Update
a26a03f to
b0dad71
Compare
clonker
left a comment
There was a problem hiding this comment.
Just some smaller nits, I think this is in good shape!
| { | ||
| Operation operation; | ||
| char const* name; | ||
| /// How often the operation is drawn, relative to the others |
There was a problem hiding this comment.
is that describing a discrete distribution where the probability to draw it is the weight over the sum of all weights?
There was a problem hiding this comment.
Yes. But not great. So I am now adjusting it to be precentages, and adding a static_assert that they must add up to 100. Hopefully more readable? I like relative distrib, but I maybe these percentages are easier on the eye? Had its downside, but anyway.
There was a problem hiding this comment.
I mean it was just a question out of curiosity :) I am fine keeping it like it is as well. Perhaps with a tiny comment that this indeed what it does.
| static_assert( | ||
| std::size(operationInfos) == numOperations, | ||
| "Every operation needs a row in operationInfos." | ||
| ); |
There was a problem hiding this comment.
would it make sense to derive numOperations directly from operationInfos? then this assert could go. although it also doesn't really hurt, just one more thing to keep in sync.
There was a problem hiding this comment.
I think we'd need reflections? C++ has no way to count an enum's enumerators, so numOperations has to name the last one. Counting them automatically needs C++26 reflection, right? I am not sure how else we could do it. I think I'm not a C++ magician enough to do this right without reflection.
There was a problem hiding this comment.
Maybe i'm missing something but how does numOperations = std::size(operationInfos) need reflection? :)
There was a problem hiding this comment.
The catch is that GeneratorStatistics needs the count in the header, so the table would have to move there as inline constexpr. I am completely fine to leave as is if you'd rather keep the table in the .cpp.
There was a problem hiding this comment.
Yes, true. I did it now :)
|
|
||
| /// @returns a number below @a _count. | ||
| /// Draw into named variables: the evaluation order of function arguments is unspecified. | ||
| std::size_t pick(std::size_t const _count) |
There was a problem hiding this comment.
I think you had the right idea. This is only meaningful for _count <= 256 if the input is a single byte. I'd add an assert instead of changing the signature though:
yulAssert(0 < _count && _count <= 256, "pick draws from a single byte");| } | ||
|
|
||
| /// Chains of calls multiply the execution time. A function can therefore only call functions that | ||
| /// call none |
There was a problem hiding this comment.
this contradicts itself with maxCallDepth being 2 right?
There was a problem hiding this comment.
Sorry, this remained from an old version of this system. It's been fixed now to read:
A function makes at most maxCallsPerFunction calls, and no chain of calls is longer than maxCallDepth
| /// so what is known about them survives the stores through locations. | ||
| /// 2**256 - 3 .. a location minus 1 to 3, wrapped around below zero | ||
| /// | ||
| /// Limitations: |
There was a problem hiding this comment.
Could the limitations list also say that the checked facts are those of a standalone analysis of the final code, not of the analyses inside the steps?
I think that is worth adding since the stale bookkeeping bug class depends on the order of queries.
There was a problem hiding this comment.
It's true of course. It's actually kinda annoying. We'd need a property analysis for the facts while the data flow analyzer is running... that's a different beast. Anyway, yes, added it to the DataFlowAnalyzerPropertyTest.cpp's Limitations part.
Adds a FuzzTest property test,
DataFlowAnalyzerProperty.AnalyzerFactsHoldAtRuntime, that checks the facts theDataFlowAnalyzer/KnowledgeBaseclaim are true at runtime.It generates a Yul program, runs a random optimizer step sequence on it, turns every claimed fact (known storage/memory contents, variable values, constant offsets, disequalities) into a runtime check, and executes the result in the Yul interpreter. Checking the claims directly catches unsound facts even when no step exploits them observably. It would have caught the Knowledge Base Stale Bookkeeping bug fixed in 0.8.38.
Supporting changes
test/tools/yulFactAuditor: the instrumentation library.yulopti --auditprints the instrumented code;yulrun --calldata <hex>runs it on a failing input.*Solovariants of CSE,EqualStoreEliminator,Rematerialiser,LiteralRematerialiserandExpressionSimplifierin yulOptimizerTests, with regression tests for that bug.<vector>includes inMultiUseYulFunctionCollector.handSSACFGTypes.h.PS: Two upcoming PRs build on this Yul program generator: one checks that optimizer steps leave the code in the shape they promise, the other checks what
SideEffectsCollectorandControlFlowSideEffectsCollectorclaim against execution.