Repository navigation
"RangeError: Maximum call stack size exceeded" When concatenating to a function with a large array #389
Description
EDIT: I tested recursive functions and they seem unaffected by this. It seems isolated only to Enum.filter.
Maximum callstack size on chrome is 10402. Calling Enum.filter on collections of size 1000+ ends up crashing with this
Uncaught (in promise) RangeError: Maximum call stack size exceeded
I think its to do with how the code got generated.
def react_dom([]), do: []
def react_dom([nil|t]), do: react_dom(t)
def react_dom(dom) do
[tag|dom] = dom
tag = fix_binary(tag)
{attributes, dom} = case dom do
[] -> {%{},[]}
[map|rdom] when is_map(map) -> {fix_attributes(map),rdom}
_ -> {%{},dom}
end
case dom do
[v] when v != nil and (is_atom(v) or is_binary(v) or is_integer(v))->
v = fix_binary(v)
{tag,attributes,v}
children when is_list(children) ->
children = Enum.filter(children, &(&1!=nil))
children = :lists.map(&(react_dom(&1)), children)
{tag,attributes,children}
end
end
The very end has a filter children = Enum.filter(children, &(&1!=nil)), it crashses inside the filter.
const [children1] = ElixirScript.Core.Patterns.match(ElixirScript.Core.Patterns.variable('children'), Elixir.Enum.__load(Elixir).filter(children0, (...__function_args__) => {
function recur(...__function_args__) {
let __arg_matches__ = null;
//CRASH IS INSIDE HERE START
if ((__arg_matches__ = ElixirScript.Core.Patterns.match_or_default([ElixirScript.Core.Patterns.variable('x1')], __function_args__, (x10) => {
//CRASH IS INSIDE HERE END
return true;
})) !== null) {
let [x10] = __arg_matches__;
return x10 != null;
}
throw new ElixirScript.Core.Patterns.MatchError(__function_args__);
}
return ElixirScript.Core.Functions.trampoline(new ElixirScript.Core.Functions.Recurse(recur.bind(null, ...__function_args__)));
}));It seems as if 10+ callstack frames are being allocated per Enum.filter iteration there.
tenk = Enum.reduce(1..500, [], &(&2++[&1]))
filt = Enum.filter(tenk, &(&1!=nil))
:console.log("okay!")
tenk = Enum.reduce(1..1000, [], &(&2++[&1]))
:console.log("okay2!")
filt = Enum.filter(tenk, &(&1!=nil))
:console.log("not okay :(")
See for yourself with the directly above code.
Activity
[-]Recursion fails on large collections, danger ahead[/-][+]Enum.filter fails on large collections, danger ahead[/+]bryanjos commentedon Sep 7, 2017
It's possible it is hitting some recursive function in JavaScript, but will have to do more investigating to know for sure
bryanjos commentedon Sep 9, 2017
The problem here is specific to concatenation using the current function. Since the call itself is not in what ElixirScript considers the tail position, it causes a stack overflow. The code is in the generation of
Elixir.Enum.filter_listbryanjos commentedon Sep 9, 2017
The error can be resolved with a better way of prepending the head of an item to a list if the list is being generated from a function.
To make it clearer, in Elixir it's this line:
https://github.com/elixir-lang/elixir/blob/75150ec7eb7bb373fdf3c188eb01f0c002035cfe/lib/elixir/lib/enum.ex#L2879
vans163 commentedon Sep 9, 2017
The problem is the AST generated from this [head | filter_list(tail, fun)], when compiled to JS right?
Just needs to transcompile better?
Why I ask, because rewriting Enum.filter/2 for ElixirScript would not fix the core problem?
bryanjos commentedon Sep 9, 2017
Correct. I'm thinking that rewriting concatenation when a function is last will help.
Still using the breaking case, this:
Can instead be translated into this:
ElixirScript.Core.Functions.concatcan then be updated to call the function if given and append the result. Otherwise, concat as normal.It's just a thought, but I'll have time to test it out later
bryanjos commentedon Sep 9, 2017
No success with that approach 😞 . Posting the entire error here.
[-]Enum.filter fails on large collections, danger ahead[/-][+]"RangeError: Maximum call stack size exceeded" When concatenating to a function with a large array[/+]