BWETTLAUFER wrote on May 14^th, 2014 at 6:21pm:


Yes. Trying pinging one of your other machines with ping -l 513. The first part of this article may explain: http://www.draytek.com/index.php?option=com_k2&view=item&id=1415&Itemid=1581&lan...

Quote:


Can you tell me more about your server machine? How many cores, how many users, how much memory, are you running any firewalls?

Running a mass update, for example, isn't going to create large network traffic (large singular chunks) as much as it is going to create a lot of small packets. Which leads me to believe that the problem is likely related to the CPU or memory being overtaxed or too many clients are trying to use the same commands or records at the same time. ...maybe.

In any case, there is a program (probably already installed) on Linux called "System Monitor". It can tell you a lot about what is going on when the going ons are going. It should be able to tell you about net bandwidth, memory use, CPU use, and etc.

Unless you have "-server_dethread" turned on (do you?) you should be getting CPU usage for each command you are sending over. If two or more users are trying to run the same command or use the same resources (usually records) they will have to wait for each other.

Most of the commands run very rapidly and a single user action is usually comprised of quite a few server commands - with a few exceptions. Reports and large table-view sub-forms get their data in large chunks that may take a while for the server to send to the client. If you have several clients all trying to get large data at once, they can end up waiting on each other. That would create the pattern where out of several cores, only a few are being used. You can test this by having one client do as they normally do, and then have a few clients step through small sample applications with no sub-records. You should then see your CPU core usage expand.

There is a possibility that you are already seeing that here, though. Each command that gets sent to the server gets assigned its own "thread". Each thread, if one is available, gets set to a different core. But most of these server commands are very fast and small, running too quickly for the system monitor to register it as much activity at all. A few of the commands, on the other hand, are much larger and require a lot of server CPU activity over seconds or minutes. These would be seen as using many "CPU seconds".

Can you describe the "topology" of your application? Does it have a lot of big sub-form tables?

Is the notes sub-form a form or table view?

Nevermind, I see that it is a table. How many elements in the sub-form?

How are you "creating" new records with XPost? Do you mean "XResultSet"?

Looking at your system monitor picture, it appears that you are using 1.6 gig of memory (out of 12). Is most of that being used by Sesame. One of the other tabs "Processes" will allow you to monitor memory use by process.

There are many Linux settings that affect network. But most of them default to good values or are auto-tuning. So unless someone has been changing them, they are probably happy numbers. Here is an article about tuning the net for a high speed network: http://fasterdata.es.net/host-tuning/linux/

Be carefully, much about network tuning is counter-intuitive. For example, setting the tcp/ip packet size larger sounds like a good idea in that each packet will then have a higher data to header ratio. But, if you set it higher than the ethernet (hardware) packet size, the data will get split between two or more ethernet packets and one may get sent nearly empty. Also, because of the nagle algorithm, the system will wait longer to send a packet because it takes longer for each packet to fill.

You might be having a network problem, and your statement about ping failures whenever using your switch, indicates that it is likely. Can you swap out the switch for another and see if that changes anything?

But, on the other hand, the system monitor of your network looks very content. The same image for your CPUs does not look so happy. If I were forced to guess, I'd say you have CPU congestion -- multiple threads waiting for a smaller number to complete. This is likely caused by the design of your application and the way it is used.

I could do some analysis on your system, but I am afraid that that can be somewhat disruptive. Steve-in-Texas, has a similar Linux based setup (much faster than Windows) with a large application and quite a few users. He allowed me to send him "debug" versions of Sesame, so that I could track down problems he was having. You might want to consult with him about the process and the business implications of going that route.

If that is out of the question (in most businesses that sort of disruption is too difficult), I'd be happy to take a look at your application (with suitable non-disclosures if need be) and see if I can find where the slow-down is occurring. If it is something in Sesame, find and fixing these issues benefits us both.