SFTP folder listing problem (a lot of files)

Hi I have an issue with my site.
There is a folder there that have 500,000+ files , so when I connect to it and try to list folder using SFTP protocol the folder listing starts, downloads stuff but then FleshFXP just freeze after downloaded few megas of folder listing.
It's not totally freezen , since from time to time it I alt-tab it does refreshes itself (from being in "whitish" blured "program not responding" state) , once in couple minutes but this looks really bed.
Sometimes after long wait I get something :
'[R] (debug) Timer Timeout Triggerd, but timer not active! Set at: 41878328 Off at: 41255156 Now: 41878359 HWND: 789540 SOCKET: 1840 TYPE: 0'
sometimes its just froze.

So we actually have to problems:
1. Can't get folder listing
2. Program becomes unresponsive after large files list over SFTP.

Also in general if I have let's say 200,000 of files it becomes rather unresponsive and slow.

Another thing I noticed that while working over SFTP , if I let's say delete several folders, it "maps" all the folders content first by going into them and listing and only after it mapped all the folders it starts to delete.
Not sure if it's a smart thing as it waists memory , I think the better strategy would be "map folder", delete content , delete folder, go to next - would save some memory.