Hot Topic (More than 10 Replies) Re: What happened to the speed (Read 4254 times)
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Feb 12th, 2004 at 10:04pm
Print Post Print Post  
Walt, you will find discussion about speed issues and its remedies in the following link,

http://www.lantica.com/Forum2/cgi-bin/yabb/YaBB.pl?board=gen_disc;action=display...
  
Back to top
 
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #1 - Feb 12th, 2004 at 10:12pm
Print Post Print Post  
Walt, did it take that long to perform xlookup or just getting it to retrieve spec?
  
Back to top
 
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #2 - Feb 12th, 2004 at 10:23pm
Print Post Print Post  
Sesame compiles all of the form programming when the form opens. This prevents us from compiling or interpreting the programming while the user is trying to actually interact with the form. It also allows Sesame to know if there are any problems or syntax errors in the program before any of it runs.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #3 - Feb 12th, 2004 at 10:31pm
Print Post Print Post  
Mark,  Is this one time after one has changed the programming code or this is everytime one gets to retrieve spec?
  
Back to top
 
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #4 - Feb 12th, 2004 at 10:53pm
Print Post Print Post  
What could be the reason why it has been taking so long? Is it the size of the code? Is it the size of file? What are the determining factors as to how long it takes to load the file in retrieve spec? What can we do to shorten that time on the user's end?
  
Back to top
 
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #5 - Feb 12th, 2004 at 11:23pm
Print Post Print Post  
Is there are a particular reason to open and close the form often? Open the form and leave it minimized on your desktop. If you are working in other programs, WIndows will swap out the memory Sesame is using, and it will have little to no impact on the performance of your other programs. Not being a DOS program there is no reason not to leave it running. Many DOS programs, including Q&A, can impact the performance of other applications.

All design decisions are a trade-off. Any action a computer takes is going to take (x) amount of work. That can either be done all at once before it is immediately needed, or done piecemeal during interaction. Being client / server and multi-user, it is generally preferred that there be a singular operation, as opposed to a generalized slowdown. Additionally, if we allow some of the programming to run, but there is a problem later causing other portions of the program to not run, there is significant opportunity for unnoticed data corruption.

Q&A gets away with some of this by not letting you out of the programming editor until all of your code is "okay". A "feature" that has generated more than a few complaints. We let you leave the editor and even open and use your form, while you are still developing an unfinished program. This makes development and testing significantly easier.

To speed up opening your form (1 1/2 minutes is unusually long. We have a few Q&A databases here with the max allowed amount of programming - and none of them take more than 10 seconds to open), take a look at your programming to see if you have unnecessary fields in place of variables, if you can consolidate the code into fewer events, if you can eliminate repeated fields. Often whole chunks of Q&A programming can be eliminated with one well placed built in function. Additionally by using real loops, variables, centralized subroutines and functions, the code will also run much much faster. By consolidating code under fewer events, you will speed the compiling process and the running time. You will also produce much easier to read and maintain code, in that it won't be scattered across so many fields and events.

Also, does your programming produce the error dialog indicating:

"Some or all of the programming in this form (Form Name) has failed to compile. No programming will run."

Programs that have errors take much longer to compile because they have to log the errors, and cleanup before compiling the next program.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #6 - Feb 12th, 2004 at 11:41pm
Print Post Print Post  
When you run "test" inside SDesigner's programming editor - how long does it take? Does it produce warnings or errors?
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #7 - Feb 13th, 2004 at 2:06pm
Print Post Print Post  
Quote:
With the new features in Sesame I suppose changes can be made to improve performance.
What worries me is that even if it can be improved by as much as 50%, it'll still be too slow to be usable.
What takes place immediately in the old "DOS is dead" QA5 takes 1 1/2 minutes in its "successor".
Hope I'm wrong as I was really looking forward to using it.


Sesame provides programming abilities that can allow you to take tremendous amounts of overhead out of your programming and databases. We have subroutines to get rid of repetetive code, variables to get rid of "programming only" fields, etc. Extra programming takes time to compile. Extra elements take time to gather information about and draw on the screen. You may be able to do much better than a 50% improvement.

In addition, for your particular issue of making people wait while you look up the record, you do have a couple of other alternatives:
1. Open a Search Tab and leave it open. The programming only compiles when you first open it. That way, your Search tab will be open and ready to go when Customer asks. In Sesame, you can add new records onto the end of a retrieved Result Set, so this might cover your data entry needs also.
2. Make another Form on your database that is not so heavily programmed and use it to Search.

  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #8 - Feb 13th, 2004 at 2:59pm
Print Post Print Post  
Studied the explanations given by Mark and Erika. In Q&A compilation is performed while closing the program editor and if something is wrong, one has to correct it before it can be saved and compiled. In Sesame, even if something is wrong, it will let one save whatever is done by giving explicit message that it could not be compiled fully and let you work with it.

In the translating phase (QA to Sesame), we expect to have some editing and modification done, so this feature in Sesame is useful. But as a trade off, it need to re-compile the form (database) every time we open the form. For heavily programmed and long forms applications, especially when you have to open and close the forms every time you serve the customer is too much wastage of time and not practical.

Why it cannot be something like,  when it detects that something is changed in the programming module since the last use or when error was detected on the last run, it recompiles. If there was no error in compiling in the last run and no programming update, there is no need for recompiling.  This way the trade-off is more tolerable and sensible.  Programming change after stabilization of program may be done once in a blue moon and for that you pay the price every working minute of the day. It sure does not make a lot of sense to me!!!  
  
Back to top
 
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #9 - Feb 13th, 2004 at 3:32pm
Print Post Print Post  
Quote:
Additionally, if we allow some of the programming to run, but there is a problem later causing other portions of the program to not run, there is significant opportunity for unnoticed data corruption.


In order to address the data corruption, the checking and recompilation can be done during the first run of the day.
  
Back to top
 
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #10 - Feb 13th, 2004 at 4:12pm
Print Post Print Post  
Quote:
Why it cannot be something like,  when it detects that something is changed in the programming module since the last use or when error was detected on the last run, it recompiles.


Detecting that it is changed / unchanged, requires that at some point the program mark it as unchanged. In a client/server environment that is not logically possible between runs of the application unless we track every user that may ever open that form, and list whether they (individually) have seen the change, so that the system can mark that program as "unchanged" for that user.

If you mean that they (the users) keep the application open, but close the form over and over, each client can then track whether they have compiled that form's programming. They would have to compile it on the first open. But if they are willing to keep the application open, why not simply leave the form open? I can probably implement something like this. I'm looking into it. In the meantime, the work around is very simple - leave the form open.

Q&A users were taught over the years that they should leave Q&A down when not in active use. With Q&A this is especially important because it leaves the file in a "open" state while Q&A is "up", meaning that a network/power/systemic failure is likely to corrupt the open file. Sesame, on the other hand, loads files and does not access them in any way unless there is a change in data. The file is kept closed unless there is activity. Unless Sesame is actively writing to the file, which only occurs immediately after a change to a form, you can pull the plug or crash your OS, and the only result in the Sesame .db/.dat file is that it will need to be "unlocked". This is especially important in a cleint/server environment, where customer expectation is that a server will be continuously running - with the applications loaded and ready to access.

As some of the Beta testers know, we used to compile code as it was needed. But there were objections that because some of the code may fail to compile, while other portions would compile and run - that invisible data corruption (wrongness) could occur. So, I moved all the compilation to a central place (opening the form), and refuse to run any code if any portion of it is "bad".

I am on the fence about this issue. On one hand, I see the refusal to run any code because a portion of it will not compile, as hampering development and testing, in which case you may want to get a portion of your code up and running, before completing and debugging another section. But I also don't want to be in a position where some part of the code is failing and the data is left incomplete.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #11 - Feb 13th, 2004 at 4:47pm
Print Post Print Post  
Quote:
If you mean that they (the users) keep the application open, but close the form over and over, each client can then track whether they have compiled that form's programming. They would have to compile it on the first open. But if they are willing to keep the application open, why not simply leave the form open? I can probably implement something like this. I'm looking into it. In the meantime, the work around is very simple - leave the form open.


Thank you very much for your response. Still I do not have proper crystallization and proper understanding. As a user and novice at that, I naturally would not understand the complexity that goes into networking aspect of it.

I know that I will have multiple forms and multiple databases in one application and will have several such applications that I will be working with in the networking environment everyday. How many forms and applications should I leave open to save time, I believe, will get clear once I put everything together.

After preparing record for a client, I have to go to retrieve spec number of times to print out various documents, sometimes on the last client seen, while other times some other clients seen earlier.

So long as application is not closed, even though  search menu has been closed, and if it does not need to recompile everytime you go to search menu again, will definitely a step in the right direction and a great help. Thank you for taking time to explain this to us.
  
Back to top
 
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #12 - Feb 13th, 2004 at 4:56pm
Print Post Print Post  
Quote:
I am on the fence about this issue. On one hand, I see the refusal to run any code because a portion of it will not compile, as hampering development and testing, in which case you may want to get a portion of your code up and running, before completing and debugging another section. But I also don't want to be in a position where some part of the code is failing and the data is left incomplete.


Could it be, that incompletely compiled application will run in the preview mode of Sdesigner but would not run in the .db (Sesame runtime) unless it is compiled completely and cleared of all errors?

In other words, save as and merge application commands will fail if application is not fully compiled and debugged.
  
Back to top
 
IP Logged
 
The Cow
YaBB Administrator
*****
Offline



Posts: 2530
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #13 - Feb 13th, 2004 at 5:26pm
Print Post Print Post  
Quote:
Could it be, that incompletely compiled application will run in the preview mode of Sdesigner but would not run in the .db (Sesame runtime) unless it is compiled completely and cleared of all errors?


Maybe do-able.

Quote:
Thank you very much for your response. Still I do not have proper crystallization and proper understanding. As a user and novice at that, I naturally would not understand the complexity that goes into networking aspect of it.


Lets say that we have an application with one form. The designer changes the programming on that one form. Three users use this form. Sesame would have to track which users have seen the form and recompiled the code. With three, and only three users, that may not be a big problem. But if more and more users get to use the form, or the form has no login, or the application is on a web server and the users are anonymous - it gets to be impractical.

One solution is to compile the code during design time and store the compiled code on the server. Then when a client accesses that form, they are sent the compiled code. The problem with this is also performance in that the compiled code is much larger than the uncompiled "source" code. So sending it to the client would take time - probably time while they are opeing the form. The other problem is that the compiler in SDesigner plays by slightly different rules than the compiler in Runtime. In Sdesigner you can reference a LE that has been "ghosted" (dragged off the form entirely). Such LEs are not visible to the compiler at Runtime.
  

Mark Lasersohn&&Programmer&&Lantica Software, LLC
Back to top
IP Logged
 
BOBSCOTT
Senior Member
Members
*****
Offline


That Darn Computer #$X#
{curse words}

Posts: 1195
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #14 - Feb 13th, 2004 at 7:34pm
Print Post Print Post  
Quote:
I guess until Tom writes another Bible I'll be bouncing off walls.


I sure agree with you on this one. I hope someone puts something out soon! Smiley

At this point I would settle for Ray the reapers or Bharats notebook.  Grin
  

Team – Together Everyone Achieves More
Back to top
 
IP Logged
 
Bharat_Naik
Senior Member
Members
*****
Offline


Ever ready to learn and
share

Posts: 1202
Location: Chicago,  Illinois
Joined: Dec 16th, 2003
Re: What happened to the speed
Reply #15 - Feb 13th, 2004 at 8:04pm
Print Post Print Post  
Quote:
FYI this is an invoice db with 1005 fields.
Part of it consists of 51 lines (17 per page) each with 9 fields left to right per line (SKU, QTY, Backorder, Shipped, Desc, etc)

Server Admin Error Log shows:
246 lines of "Element name resolves to a number"
4 lines for fields named %, $, #, and @15%
1 line for "[Amount^] contains operator characters" (even though there's no field named Amount)
1 line for "[S88] cannot be retained with value [____]"
1 line for "[P1] cannot be retained with value [_____]"



Walt,
You sure know how to strain the system! I am glad you are not the coach of my kid's football team!!!  Smiley   Grin
  
Back to top
 
IP Logged
 
Ray the Reaper
Global Moderator
Members
Lantica Support
*****
Offline


The One & The Only

Posts: 2482
Joined: Aug 20th, 2003
Re: What happened to the speed
Reply #16 - Feb 13th, 2004 at 8:10pm
Print Post Print Post  
Quote:
1 line for "[S88] cannot be retained with value [____]"
1 line for "[P1] cannot be retained with value [_____]"
(both the S88 and P1 fields above contain numbers, so why this msg?)


The field names contain numbers but there is probably a record in your Q&A database that has the data "____" in those fields. The errors are telling you that it is clearing those fields rather than have the junk data in them
  

Raymond Yoxall Consulting
ray.yoxall@gmail.com
ryoxall@lantica.com
Sesame Applications, Design and Support
Back to top
IP Logged
 
josebetzy
Member
*
Offline


Hmmm. How? Cómo?

Posts: 33
Location: Las Piedras, Puerto Rico
Joined: Dec 15th, 2003
Re: What happened to the speed
Reply #17 - Feb 14th, 2004 at 11:45pm
Print Post Print Post  
Just listening to the speed problem, I have a QA5 translated database with 10,230 records, 198 fields (elements) plenty of programming, lookups. When I first open the application for the first time, it takes about 27 seconds, then I have no problem. I select the database, retrieve (F7)  whatever and get the results in less than a second. I have tried using several levels of retrieve at the same time for very specific records and it gets them in a blink. The only complaint I have is the Table View, but that has a solution in better design.

The only help experience I can give is that after cleaning up the QA5 database before translation, has been the solution to speed problems. I had a very early post that it was slow. I followed the Lantica recommended translation checks and the improvement was very significant.  I hope this helps because I do not think I am unique.
  

Jose L. Muñoz
Back to top
 
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: What happened to the speed
Reply #18 - Feb 15th, 2004 at 1:49am
Print Post Print Post  
Quote:
FYI this is an invoice db with 1005 fields.
Part of it consists of 51 lines (17 per page) each with 9 fields left to right per line (SKU, QTY, Backorder, Shipped, Desc, etc)

This sounds like a perfect example of a Q&A database that should be broken into subforms.  It sounds like you have all these pages for line items on the invoice.  Each of these lines should really be a separate record in their own database.  shown as a sub form to the Invoice form.  The other 138 fields are probably associated with the Invoice "header", common to all line items on the invoice.

I would even suggest you might want to do the redesign in Q&A before translating to Sesame. Make copies and work on the copies only.  It will take some effort, but make a new InvItems database, and be sure to use a Unique field for each record, most likely the Invoice number.  Copy the Invoice Number and all the info on line 1, for all the records with info on line 1 to the new database, then copy the Invoice Number and all info on line 2 for all the records with info on line 2, etc.  Do this for all 51 lines.  Now delete all 51 lines from this database and all you should have are the 138 remaining fields which will be the Invoice Header.

Now when translating into Sesame, bring both Q&A databases into the Invoice.db application.  Bring in only the Header information to the InvHeader database and bring in InvItems into the InvItems database.  Then make the InvItems a subform for Invoices form that shows the InvHeader information, linked by the Invoice Number field to show the InvItems records.

Note, the suggestions are a concept only, and the specifics may need to be altered based on the contents if the Q&A  records.  Example, maybe you can't use Invoice number as the common field, maybe you need something different like data and/ or time and/or record and/or Customer's PO, etc.
  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #19 - Apr 8th, 2004 at 10:34pm
Print Post Print Post  
Walt,

In order to get from a flat Q&A file into a naturally linked form/subform in Sesame, you'll have to do it in steps. Q&A has no way to organize or export the info in such a way that Sesame knows what you want, so you need to use Sesame to do the reorganization.

1. From Q&A, export only the Header info from the Invoice records. Make sure you include a single field that identifies the invoice.

2. From Q&A, export each "row" of line item fields from the Invoice records. Make sure you include a single field that identifies the invoice itself.

3. Use the operating system or Notepad to tack all the line item files together into a single file. Do not concatenate in the file with the Header info.

4. In Sesame, build your Form and subform, relationally linked by whatever field you exported to identify the invoice.

5. Import the Header records to the Invoice database.

6. Import the Line Item records to the Line Item database.

7. You should now have relationally linked invoices with all the line items. If you wish, you can follow the steps detailed in another thread to change over to a natural link.
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Carl Underwood
Senior Member
Members
*****
Offline



Posts: 1351
Location: New Hampshire
Joined: Mar 11th, 2003
Re: What happened to the speed
Reply #20 - Apr 9th, 2004 at 3:06am
Print Post Print Post  
Erika wrote Quote:
In order to get from a flat Q&A file into a naturally linked form/subform in Sesame, you'll have to do it in steps.

Something that worked for me was to simply export all the info from all the fields in the Q&A DTF at once, into one export file. Then I selectively imported from that file with mutiple imports using [Ignore Column] repeatedly so that I could pluck just the fields I wanted on each pass.

This saved me from manually editing in Notepad.

(Hope I didn't forget something, it's late and I'm pulling this from memory out of a very sleepy brain. Smiley)
  


Carl Underwood
CDU Computer Consulting LLC
Epsom, New Hampshire
Back to top
IP Logged
 
Bob_Hansen
Senior Member
Members
*****
Offline


WOW, They have the Internet
on computers now!

Posts: 1861
Location: Salem, NH
Joined: Nov 24th, 2002
Re: What happened to the speed
Reply #21 - Apr 9th, 2004 at 3:56am
Print Post Print Post  
I did multipe copy/exports.  One pass for each line.  Revtrieve all records where linex is not blank.  Copy/Export the  Invoice(Key) value, plus all of the items for that line, including a line number.  If no line number existed before, create a field for each line and do a mass update to create the line numbers before trying to take out the data.

Do one Copy/Export for alll of the header (common) information on all records.  Will normally have to modiry process for totals of line items, etc..  But since this is being brought into Sesame, only need to have the fields in place, will do the programming in Sesame to replace the XPosting type of steps this would have required in Q&A.

Endup with two databases:  Invoice Header, and Invoice Items, one record for every line item that was in original database.  Still have the original database intact.  And also have a backup that I made before this whole process.

Should now be able to bring into Sesame with either natural or relational links.
  



Bob Hansen
Sesame Database Manager Professional
Sensible Solutions Inc.
Salem, NH
603-898-8223
Skype ID = sensiblesolutions
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #22 - Apr 10th, 2004 at 7:59pm
Print Post Print Post  
Quote:
Hammer, Carl, Bob,
Thanks for your help!

Erika, the steps you outline end up with relational links, which would then need to be converted to natural for faster performance.
However, couldn't I go directly to natural link by creating naturally linked empty form/subform, then importing the complete ASCII file including both header and line items in a single pass per page 322 of the User Guide?


You could, but you would need to do some pretty hefty munging of your header and line items files to get them into a single file of the correct format for the hierarchical import. If you do the relational link, then export the data from Sesame, it will create the hierarchical import file for you. You can then switch to natural links, import the exported file, and you're good to go.
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged
 
Hammer
YaBB Administrator
Lanticans
*****
Offline


Fire bad. Tree pretty.

Posts: 3436
Location: Ohio
Joined: Nov 22nd, 2002
Re: What happened to the speed
Reply #23 - Apr 11th, 2004 at 12:43pm
Print Post Print Post  
Quote:
Thanks Erika,

Forgot that I can't export the required hierarchical ASCII file from the Q&A database.  Do you have any ideas on how to make Q&A fake it?  Sure would save a ton of time.


If I did, I'd have told you in the first place.  Grin
  

- Hammer
The plural of anecdote is not data.
Back to top
IP Logged