MS Access Forum / Multiuser / Networking / June 2006
multiple users writing records to the same table at the same time
|
|
Thread rating:  |
Jim Franklin - 24 May 2006 12:54 GMT Hi,
I am working as part of a team which has an Access 2003 application, whereby 3 or 4 users could be simultaneously running processes which write large numbers of records to a single log table. Because of the nature of the process, each record is written individually in VBA, rather than using a SQL transaction. Each process opens and closes the log recordset several times during the course of the process.
As each process can write up to 1,000,000 records, we are getting various errors & access lockups because the file is locked to other users.
Does anyone have any useful information or pointers as to the best approach to handle this and avoid this problem?
Thanks for reading, and for any help anyone can provide,
Cheers, Jim F.
Jim Franklin - 07 Jun 2006 12:58 GMT Is there anyone who can help me with this? Please!!!
(Desperate now!!!) Jim F.
> Hi, > [quoted text clipped - 15 lines] > Cheers, > Jim F. Sylvain Lafontaine - 08 Jun 2006 01:38 GMT In the Options | Advanced tab, make sure that you have chosen to open databases using record-level locking; this might help (but I don't really know for sure).
What do you mean by SQL-Transaction? Are you talking here about a JET Workspace or if you are using SQL-Server as the backend? Even when you are writing single records, you should use a transaction if you are using a JET Workspace as this may help for locking problems.
Otherwise, with multiple processes writing 1,000,000 records and if you are using JET, then maybe it's time to replace it with SQL-Server as the backend database. SQL-Server offers a lot more than JET in term of control over locking and dead-locking mecanisms.
Finally, you don't say how you are making your updates. With all the possibilities, it's always hard to guess what someone is doing exactly and how the various solutions might apply.
 Signature Sylvain Lafontaine, ing. MVP - Technologies Virtual-PC E-mail: http://cerbermail.com/?QugbLEWINF
> Is there anyone who can help me with this? Please!!! > [quoted text clipped - 20 lines] >> Cheers, >> Jim F. Larry Linson - 11 Jun 2006 01:48 GMT With that many records, are you certain you just haven't run up against the 2GB database size limit? Are you regularly doing "Compact and Repair"? Right-click the file and look at the size. Is your database split into front-end on the users' machines and back-end on a server? If I recall correctly, _adding_ records to a table does not cause locking problems -- they were clever enough to fix that problem several versions back. Are any of those log writes _updates_? That's when you are likely to encounter collisions.
Are you, in fact, writing to a Log Table in the database, or to a Log File that is on the disk but not in the database?
Larry Linson Microsoft Access MVP
> Is there anyone who can help me with this? Please!!! > [quoted text clipped - 20 lines] >> Cheers, >> Jim F. Jim Franklin - 12 Jun 2006 16:02 GMT Sylvain, Larry, David - thank you all for your help. I really do appreciate it.
To answer your questions all in the same place...
Sylvain, by SQL-Transaction, I mean running a SQL string to insert a whole batch of records together with an INSERT INTO statement. What we are doing is opening a recordset in vba and adding new records individually using .Addnew and .Update etc. What we have is a system which, for example, builds an updateable recordset, and loops through it, record by record, applying updates according to rules data held in another table. Each update triggers the addition of a new record in our log table, held in its own Access db on the network server. There are often several thousand updates per process, and a handful of people might be running processes at the same time. I know it would be much easier and faster to execute a SQL statement, but for various reasons, we can't.
Larry, we are using a common FE/BE configuration, as you say with the FE on each user's machine. The 2GB size limit is not an issue, and there are NEVER any updates done to the log table, only additions as described above.
David, although we have had a number of different error messages, with similar results, the most common one is err.number 3011 (db is read-only) - once this occurs for one person, no one can write to the BE log db. The only table in the log db is a single log table with 6 fields. Most of the fields have indexes on them, and the primary key is a multiple field index.
We are not using transactions (although maybe we should be?) as we are opening and closing the log recordset every time we add a record.
Hope this helps! Any feedback is greatly appreciated! Jim
> Is there anyone who can help me with this? Please!!! > [quoted text clipped - 20 lines] >> Cheers, >> Jim F. Sylvain Lafontaine - 12 Jun 2006 19:43 GMT « running a SQL string to insert a whole batch of records together with an INSERT INTO statement » is not a transaction, it's a Batch. The batch can be enclosed in a transaction or contains one or multiple transactions; however, I don't know if Access will enclose automatically a batch inside a transaction.
However, even if I replace the word Transaction with the word Batch, I still don't see why you have some problems. The records should be individually locked by Access when it's making an update but the process should be fast enough to be finished before any timeout arises from a locking problem.
Also, in older versions of Access, the maximum number of individual locks was around 10000; so if the required number of lock is too high, then Access will lock the whole table. But in your case, you are closing and reopening the recordset each time you add a record; so again, you shoudn't have any problem.
Maybe it's time for you to take a look at MSDE or SQL-Server 2000/2005 or SQL-Server Express 2005.
 Signature Sylvain Lafontaine, ing. MVP - Technologies Virtual-PC E-mail: http://cerbermail.com/?QugbLEWINF
> Sylvain, Larry, David - thank you all for your help. I really do > appreciate it. [quoted text clipped - 56 lines] >>> Cheers, >>> Jim F. david@epsomdotcomdotau - 15 Jun 2006 00:24 GMT A transaction is one way of doing a batch update. All of the records are handled locally, then at the end of the transaction, there is a batch update.
You should remove all of the indexes. Tables should be re-indexed after intensive updates. Are you using any of the indexes in your append process?
Do the records have to be interleaved? Can you add all of the records to a local table, then do a batch update with all of the new records?
Do you have a persistent connection to the BE?
If you are not using a persistent connection, is there an LDB file after lockup? Can it be deleted?
After the BE locks up, does it automatically unlock itself?
Does it help if you use application.dbengine.idle
(david)
> Sylvain, Larry, David - thank you all for your help. I really do appreciate > it. [quoted text clipped - 53 lines] > >> Cheers, > >> Jim F. david epsom dot com dot au - 09 Jun 2006 04:15 GMT What are the 'various errors' and 'access lockups'?
Please provide an example of your VBA code.
What kind of indexes are you using on the table, and why are you using indexes at all?
Are you using transactions? If so, why? if not, why not?
(david)
> Hi, > [quoted text clipped - 15 lines] > Cheers, > Jim F.
|
|
|