Call it a matter of personal tastes, but I would avoid outputline in favor of actual html with inline ampscript. at least to me, all sense of indentation is lost in your examples, and the code gets much harder to read. I think as a result, logical connections get blurred that a normally formatted HTML would visualize better.
So by restructuring the code in that manner, I think I found a reason why you do not see the intended behavior: your s and the different parts of the table are inside and outside of an IF condition, and that condition changes during your loops. So you might get stuck with unmatched pairs, e.g. an unclosed table. See below for my untested attempt. For each order with an order #, is always there. All content inside of the table(s) is shown if your prevOrder condition is met. Might not be 100% accurate but this is where I would start.
%%[
set @contactid = AttributeValue("Contact ID")
set @numRowsToReturn = 0 /* 0 means all */
set @rows = LookupOrderedRows("Order Follow Up", @numRowsToReturn, "Order Number", "Contact ID", @contactid)
set @rowCount = rowcount(@rows)
IF @rowCount > 0 THEN
set @prevOrderNo = ""
FOR @i = 1 to @rowCount do
set @row = row(@rows,@i)
set @OrderNo = field(@row,"Order Number")
/* create a table for each order */
If not empty(@orderNo) THEN
]%%
<br>
Order #%%=v(@OrderNo)=%%<br>
<table class="test">
%%[
set @itemid = field(@row,"Item ID")
set @price = field(@row,"Price")
set @qty = field(@row,"Qty")
IF empty(@prevOrderNo)
or @prevOrderNo != @OrderNo
THEN
/* if condition for order display is met, show table content */
]%%
<tr>
<td>QTY</td>
<td>ITEM</td>
<td>PRICE</td>
</tr>
<tr>
<td>%%=v(@qty)=%%</td>
<td>%%=v(@itemid)=%%</td>
<td>%%=v(@price)=%%</td>
</tr>
%%[
set @prevOrderNo = @OrderNo
ENDIF
]%%
/*close each table unconditionally*/
</table><br>
%%[
ENDIF
]%%
NEXT @i
ELSE
]%%
<br>No transactionsList rows found
%%[
ENDIF
]%%
