Patrick,
If you would open the category.php file and locate the following two lines of code:
PHP Code:
get_ids($arr_childs);
$cat[$i]['total']=get_count($arr_childs);
Then just comment them out like this:
PHP Code:
//get_ids($arr_childs);
//$cat[$i]['total']=get_count($arr_childs);
This will remove the listing count and speed the site back up. Please note those same lines will be in three or four places in the category.php file. It will also be in the index.php file.