Is there a particular order in which they are executed?
The order is undocumented, and may change from release to release or as changes to rules are made.
or is it random?
It's not "random" so much as "indeterminate." The system will run the rules in the same order for a while until conditions change, in which case the order may change. What causes the order to change is not documented.
If a record is marked as "duplicated" by a Duplicate Rule, this record will also be evaluated by the second Duplicate rules?
All rules are evaluated. It is possible for multiple DuplicateRuleSet entries to be created on a record from multiple matches. However, if any single rule matches that has a Block action, then the record is marked as "failed", and depending on allOrNone flag of the DML operatio, the remaining records will be re-evaluated, or the transaction will be completely rolled back.