fix loop protection breaking shader strings and p5.strands functions#4083
fix loop protection breaking shader strings and p5.strands functions#4083Nixxx19 wants to merge 5 commits intoprocessing:developfrom
Conversation
davepagurek
left a comment
There was a problem hiding this comment.
Great work on this! Consider this review nonblocking, just sharing some thoughts for you and @raclim to consider.
| callee.property.name === 'modify' && | ||
| callee.object.type === 'CallExpression' && | ||
| callee.object.callee.type === 'Identifier' && | ||
| /^base\w*Shader$/.test(callee.object.callee.name); |
There was a problem hiding this comment.
Might be worth noting that this might miss some things, e.g. if a plugin author exposes a shader of their own that has its own name or is just a variable and not a function. If we just check that it's a MemberExpression calling a property called modify that would capture all that, but may have some false positives if users have a method called modify in non-p5.strands code. I think either could be a reasonable compromise, @raclim let me know if you have any thoughts on which side to err on!
There was a problem hiding this comment.
good point, thank you! updated it to match any .modify() call instead of just base*Shader().modify(), which should cover plugin-defined shaders too. there could be false positives if someone has an unrelated .modify() call, but that feels like the safer tradeoff. happy to hear your thoughts on this.
There was a problem hiding this comment.
I think @davepagurek's suggestion sounds good to me! I think we can keep the check broad for now, and refine it later down the line if needed.
There was a problem hiding this comment.
thank you both for the input! will keep it broad for now and revisit if false positives come up.
| const loops = []; | ||
|
|
||
| function visitNode(node, ancestors) { | ||
| const isInsideShader = ancestors.some((ancestor, idx) => { |
| try { | ||
| ast = acorn.parse(jsText, { | ||
| ecmaVersion: 'latest', | ||
| sourceType: 'script', |
There was a problem hiding this comment.
Maybe worth considering that users could use <script type="module"> scripts, e.g. https://editor.p5js.org/davepagurek/sketches/-foIv7eQa -- worth testing to see if parsing fails in those if you use sourceType: 'script' and you have an import statement, or if we'd need to pass in the source type based on how the script tag imports it.
There was a problem hiding this comment.
thank you for catching this! added a fallback that tries sourceType: 'script' first, and if that throws (e.g. for a module script with import statements), it retries with sourceType: 'module'. added a test for this case too.
| const insertions = []; | ||
|
|
||
| loops.forEach((loop) => { | ||
| const { line } = loop.loc.start; |
There was a problem hiding this comment.
Possibly worth considering whether it's preferable to still do line-based insertions like this, which may have issues with multiple loops on the same line like what's implied by the comment in the current implementation:
p5.js-web-editor/client/modules/Preview/EmbedFrame.jsx
Lines 66 to 69 in 37bc396
...or if instead it would be better to make changes at the AST level using Acorn by modifying the AST nodes, and then use a tool like escodegen to re-emit the full source code from that. (This is what we do in p5.strands after transpiling shader functions: https://github.com/processing/p5.js/blob/9498619e7b31f1b3a4e0fda5fcdc19d38e7eff20/src/strands/strands_transpiler.js#L1772) The downside of this approach is that escodegen doesn't preserve everything about your source code like whitespace and semicolon usage, so it might look a little different if you put a debugger statement and look at the code actually being run by the browser. Also it would mean adding another library, not sure how much that adds to the bundle size. But it would mean more targeted insertions and less likelihood for it happening incorrectly.
There was a problem hiding this comment.
thank you for the suggestion! switched to the escodegen approach. we now modify the ast nodes directly and regenerate the source using escodegen.generate(). added escodegen as a dependency. the tradeoff on whitespace/formatting is real but feels worth it for the correctness guarantees. please let me know if the bundle size is a concern and we can fall back to character-offset string insertions instead, which also handles multiple loops on the same line correctly since we apply them back-to-front.
| }); | ||
|
|
||
| walk.simple(ast, { | ||
| BlockStatement(node) { |
There was a problem hiding this comment.
Do you think we can simplify this step and modify the nodes in place in the earlier pass where we collect the loops to modify? Then we wouldn't need to find them again with another pass here? Or correct me if there's something that that would miss!
There was a problem hiding this comment.
i chose the two-pass approach initially to keep the collection and injection steps separate, but you are right that since we already have the ancestors available in the walk, we can do it all in one pass. simplified it to a single walk.ancestor call that both checks for shader functions and injects the protection in place. updated in the latest commit. thank you for the suggestion!
| walk.ancestor(ast, { | ||
| ForStatement: visitNode, | ||
| WhileStatement: visitNode, | ||
| DoWhileStatement: visitNode |
There was a problem hiding this comment.
Is there anything else(i.e for...of loops) we might want to cover here? Also okay with leaving it as is!
There was a problem hiding this comment.
for...of and for...in loops are parsed as separate ast node types (ForOfStatement and ForInStatement), so they are not currently covered. happy to add support for both if you think that would be a good addition!
issue:
fixes #4080
demo:
changes:
jspreprocess.jsmodulebuild*shader()andbase*shader().modify(), second pass skips loop protection inside those functionsloop-protectanddecommentdependencies fromembedframe.jsxi have verified that this pull request:
npm run lint)npm run test)npm run typecheck)developbranch.fixes #4080