Normal Topic Need suggestion on how to import (Read 745 times)
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Need suggestion on how to import
Mar 24th, 2004 at 4:36am
Print Post Print Post  
Could someone post a suggestion on how to import only modified records into sesame from a delimited file, overwriting matching records (with same stock numbers)?

I am trying to keep a sesame database in synch with a q&a database (computers not linked together) so I want sesame to not import any records which it already has in its database. I do want it to import any records that are new or modified, overwriting any old records that have been modified.

Or is it best just to purge the sesame database before each import then I could import all records from the .txt file? Doesnt seem very efficient.

Thanks,
Steve


  
Back to top
IP Logged
 
BOBSCOTT
Senior Member
Members
*****
Offline


That Darn Computer #$X#
{curse words}

Posts: 1195
Joined: Nov 22nd, 2002
Re: Need suggestion on how to import
Reply #1 - Mar 24th, 2004 at 2:12pm
Print Post Print Post  
Steve,

Will changes be made to records in both systems that will need to be in sync. Is it possible that the same record be modified by both systems on the same day or is the QA database the only one updating and the Sesame is just displaying the data?
  

Team – Together Everyone Achieves More
Back to top
 
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Need suggestion on how to import
Reply #2 - Mar 24th, 2004 at 3:20pm
Print Post Print Post  
Bobscott,

Thanks for the reply.

Until Sesame has "proven itself", our corporate office will continue to use Q&A to daily update records. At the end of the day, they will export all new and modified records and email the records to me at a remote site.

I'll then import those records into my sesame database but will not edit any of them. I'll be creating invoices that will use xlookups to pull info from the readonly database. at the end of the day, I will export my invoices and email them to corporate who will then enter them into the database (as a modified record) and then send them back to me to go into my read-only database.

It will be a big circle of records. They only part that stumps me is how to import just records that DONT MATCH records in my readonly database.

Maybe the @isnew command would work for me, but remember, I'll be using ascii files until we give Sesame a good "workout" Smiley

Thanks!
Steve in Texas
  
Back to top
IP Logged
 
BOBSCOTT
Senior Member
Members
*****
Offline


That Darn Computer #$X#
{curse words}

Posts: 1195
Joined: Nov 22nd, 2002
Re: Need suggestion on how to import
Reply #3 - Mar 24th, 2004 at 5:00pm
Print Post Print Post  
Steve,

I always try to find elegant solutions to problems but this may not be a fight worth fighting. Since your basically replacing all the data anyway it might make more sense to just replace all the data each day. At quick glance it seems much less complicated to make some kind of a routine that replaces all the data rather then append some of the data.
  

Team – Together Everyone Achieves More
Back to top
 
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Need suggestion on how to import
Reply #4 - Mar 24th, 2004 at 5:08pm
Print Post Print Post  
I'm starting to agree with you. It was just trying to avoid transferring duplicate records via email.

In an ideal situation, I would be working directly out of the corporates database via LAN or VPN, with readonly rights, but I think that is a step to big for us to take right now.

Hopefully soon.

Thanks for the suggestion.

Steve in Texas
  
Back to top
IP Logged
 
BOBSCOTT
Senior Member
Members
*****
Offline


That Darn Computer #$X#
{curse words}

Posts: 1195
Joined: Nov 22nd, 2002
Re: Need suggestion on how to import
Reply #5 - Mar 24th, 2004 at 6:03pm
Print Post Print Post  
Quote:
I was just trying to avoid transferring duplicate records via email.


That may be a different story. I did not understand the problem before. The issue is the transfer time of the data.

In that case lets think about this.
We know:
It is easy to have a field in QA to identify what records are new or modified each day.
It is easy to export just those records from QA.

If it is a relatively small percentage of all the files so that transfer of records would be considerably shorter maybe its worth thinking about creating a  routine using XPOST.

Unfortunately I know very little about xpost but on page 38 of the Sesame programming Guide   it says posts the entry value to the target element on a match between the key element in current database and the external yada yada.. default operation is Replace.

In basic terms imagine transferring the small file then
Create a routine something like this:{I am reaching here. There may be a cleaner way to use Xpost)

import data into a temporary database,  that database uses xpost to match record id in good Sesame database and it marks a field in your good Sesame database that is used as a way to identify the record for mass deletion.

Then the mass delete is ran

Then when the exported file is imported all would be in sync.

I hope that makes some sense. The key here is how much time is saved by transferring just the modified records as opposed to all the records.

Let us know if it is a significant amount of time and if it is we can learn more about xpost or ?

Robert
  

Team – Together Everyone Achieves More
Back to top
 
IP Logged
 
Steve_in_Texas
Senior Member
*****
Offline


No personal text

Posts: 893
Location: San Antonio
Joined: Feb 21st, 2004
Re: Need suggestion on how to import
Reply #6 - Mar 24th, 2004 at 6:29pm
Print Post Print Post  
Robert,

I think you solved my delimma. There may be only 1 or 2 modified records per day, out of 20,000 total. So your right, I should not import/export/email 20,000 records just to "syncronize" the 1 or 2 that actually changed.

I like your idea of importing just MODIFIED records into a temporary sesame database, then use another step to synch my real database (readonly) with the temporary.

There will be about 15 to 20 fields total that will need to be emailed, so hopefully the xpost, or @isnew programming will not be too extensive.

Anyhow, I think you are leading me in the right direction. I'll read up on xpost.

If you have any other suggestions, please let me know.

Thanks again!
Steve in Texas
  
Back to top
IP Logged