The iteration runs on a function and it only produces one matrix 1000 x 1000.
You may well be right, but you may also be wrong. You would not be the first to get caught by what you think Mathcad is doing, vs. what it is actually doing. To clarify what Werner said, if at the worksheet level you type
A:=2
A:=A*2
The second A does not overwrite the first. You have two independent copies of A, and there is no way to subsequently free the memory used by either of them. I wrote simple expressions using only operators, but this is true regardless of how A is created or subsequently modified.
What happens in a program is rather different though. If you write inside a program:
A<--2,
then when the program exits the memory allocated to A is freed.
If you write
A<--2
A<--A*2
I forget whether the second A overwrites the first. I know it's been discussed in a previous thread, but I can't find it. I think the second A does in fact overwrite the first, but I'm not certain of that. Regardless, when the program exits the memory for all variables that are local to the program is freed.
Without seeing the worksheet it's impossible to say how this might be applied to your problem, but if you are having memory problems you should at least be aware of it.