I have two formula fields (ex. Parent and Child) and I want to recalculate the child field when the parent's value changes. Is there any way to do this? Is it possible to set the Child value from the Parent formula field? Parent formula is very complicated, so if I will use trigger it can have up to 500-800 rows and I want to avoid using a trigger.
To answer your exact questions: Yes you can reference the parent via a Formula. Yes the child can change based on the Parents formula, asuming you are referencing the parent someway in the childs formula. The challenge with a Large formula is the compile size another formula referencing it may blow out the limits. there are a few tricks you can use to reduce compile size, this is where computer science (1s and 0s within RAM or DB etc) knowledge comes in handy.
So if you have many child rows within your trigger and you need to include that heavy formula then yes it does get heavy. and hence you are best to bring the compile size down, or change the way you are processing the many rows.
if you provide the formula I can help reduce its size.
using custom labels over settings (if you must) using custom settings over lookups. not using formulas and saving the data so that it is resident on the record. there are so many tricks.
Formula fields do not "Recalculate" as such, not like a Roll Up field, they do not fire re-processing of parent data or the like. Some formulas cannot be used in Roll Ups because of the innate complexity of the formula calculates on query and a roll up processes after save.
Formula fields will always run based on the the data they process. This is why the have a compile size, the compile size is about keeping the formulas as clean a possible so that salesforce performs well during a data query (UI or coded)
This is a complicated question because you are also asking about triggering on hundreds of fields, in all honesty I think you may be misusing trigger or formula.
Think of formulas the same as in Excel they process data in a row, but can also look up 10 levels. HERE
Triggers are much more complicated and can be very complex if there is a large hierarchy of data and volumes of data, other triggers and process builders, and yes large formulas. At which point you need to understand order of execution.
If you go into more detail about what you are trying to achieve maybe we can help more.