dbappend() problem on Windows7 shared drive.

[skiman]

Hi,

In an attempt to find the cause of our index corruptions we did tests today. Below is our little test program.

When your data is on a Windows 7 station, the APPEND isn't working correctly! Even worse, you never encounter a NETERR(), but not all the records are added. This is a real problem and a bug in Xbase++. If the little program below isn't working, how can we trust our data is written correctly in our DBF files!!!

If a customer uses a shared drive on a Windows 7 PC, your are in trouble!

Code: Select all

use faktuur alias fakt new 
set index to fakt1000, fakt2000, fakt3000
// we always use NTX files
for teller = 1 to 10000
  fakt->(dbappend())
  if fakt->(neterr())
        msgbox ("Neterror bij record : "+str(teller))
  endif
  fakt->volgnr := teller
next

This is tested on a network with 9 PC's which contains of 2 XP, 4 Vista, 3 Windows 7
Test 1: (multiple times)
- Database and exe on XP station.
- Sample started on Windows 7 and Vista:
- Result: always 20.000 records, which is expected.

Test 2: (multiple times)
- Database and exe on Windows 7 station.
- Sample started on Windows 7 and Vista
- Result: NEVER 20.000 records, at least 15 records are gone, sometimes even more. There was never a NETERR() !!!!

Test 3:
- Database and EXE on Windows 7 station
- Sample started on two Windows 7
- Result: 19999 records, no neterr().

Test 4:
- Database and EXE on Windows 7 Station
- Sample started on Windows 7 and XP.
- Result: 19998 records, no neterr()

If anyone has some suggestions, please send them.

[rdonnay]

Chris -

Are you also seeing this problem with Advantage Server?

Which DBE are you using?

Before using Advantage, Bobby Drakos and I discovered a similar problem, but we were able to reproduce it with XP.

Roger

[bwolfsohn]

Chris,

interesting results...

What is the status of your oplocks settings ???

Brian

[skiman]

Hi Roger,

With a XP as shared drive we don't have this problem. As soon as the shared drive is a Windows 7, the problem comes up.

Code: Select all

DbeSetDefault("DBFNTX")
DbeInfo(COMPONENT_DATA,  DBFDBE_LIFETIME,  0)
DbeInfo(COMPONENT_ORDER, NTXDBE_LOCKDELAY, 10)
dbeinfo( COMPONENT_DATA , DBFDBE_LOCKMODE, DBF_NOLOCK )
dbeInfo( COMPONENT_ORDER, DBE_LOCKMODE    , LOCKING_STANDARD )

[skiman]

Brian,

Yes, the magical words: Oplocking.

First of all, we already did some tests with the oplocking. But at customers sites we still had problems.

This is in my program to automate the registry setting. After this, the problems remained!

Code: Select all

function Oplocking(cType)   // cType = SERVER or STATION
***********************************************************
Local oReg

if cType =="SERVER"
	oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters") 
    If ! oReg:Status()
       oReg:Create() 
    endif
    oReg:SetValue( "CachedOpenLimit", "0" )

    oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters") 
    If ! oReg:Status()
       oReg:Create() 
    endif
    oReg:SetValue( "EnableOplocks", "0" )
endif

if cType == "STATION"
    oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MRXSmb\Parameters") 
    If ! oReg:Status()
       oReg:Create() 
    endif
    oReg:SetValue( "OplocksDisabled", "1" )

    oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkStation\Parameters") 
    If ! oReg:Status()
       oReg:Create() 
    endif
    oReg:SetValue( "UseOpportunisticLocking", "0" )

    oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkStation\Parameters") 
    If ! oReg:Status()
       oReg:Create() 
    endif
    oReg:SetValue( "UtilizeNtCaching", "0" )

    oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkStation\Parameters") 
    If ! oReg:Status()
       oReg:Create() 
    endif
    oReg:SetValue( "UseLockReadUnlock", "0" )
endif
msgbox("Uitgevoerd op "+cType)

return nil

However, this is B** S**it! As it seems that this doesn't solve the problem, you can't blame it to this stupid settings. We had the following at a customer site.

Day 1: You have to set the oplocking on all your PC's. If you are working on the server, just execute the function as server and as station.
Day 2: This didn't help, so we asked if he was really sure he did it on all his stations. He did it again.
Day 3: Maybe there is a laptop connected to the network where this isn't executed, so he did this.
Day 4: Another salesman of the company came in, and connected his laptop... I can tell you what the customer said, but it isn't really for publishing. And in fact, he was right!

If the problem can really be proved with a simple sample as above, there must be something that Alaska can do with it. At this moment I'm trying with dbskip(0), dbcommit() and everything I can think to be sure the append() is really appending, before I write something into it.

The problem is that the record is added, but afterwards an append() on another station overwrites this record. I'm quite sure that this is the real problem. I don't know it this is because of the header of the DBF isn't already updated.

I will change the append() function to verify if the record is really empty after the append(). This way maybe I can see if a problem arise, and act accordingly.

[rdonnay]

Chris -

Your solution appears to be the only one that will guarantee a record is created.

That's what Bobby and I had to do.

We created a wrapper for dbAppend() and then did a dbSeek() in the new wrapper function to be sure it was added.

I don't know if it is feasible for you to use Advantage Server in your application, but we NEVER have problems with it. This is the nature of Client/Server technology.

Bobby and I met with the development team of Advantage Server here in Boise, Idaho 2 weeks ago, and I recall Bobby telling them "Ads literally saved my life". That is true. He was suffering greatly from missing transaction records until he came out here for a visit and in 2 days we had solved a problem he had been living with for months. Now he is an evangelist for Ads. On his last trip we looked at ways to improve the performance of his transaction system using SQL for the writes and some of the queries.

Roger

[Auge_Ohr]

hi

what you describe is a "Op´s Locking 2nd Level Cache" Problem

a.) what Windows 7 Version ?
Home or Pro ?

b.) what XP Service Pack 2 or 3 ?
( see KB922129 )

c.) how do you "mount" share Drive ?
NLA (Network Location Awareness) or LLTD (Link Layer Topology )

d.) what Op´s locking Setting ?
Server and Client

and dbeinfo() Setting have nothing to do with Op´s lock.
use "default" Value to get a stable Network first, than "optimize" with dbeinfo()

[Auge_Ohr]

Code: Select all

oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters") 
oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters") 
oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MRXSmb\Parameters") 

oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkStation\Parameters") 
oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkStation\Parameters") 
oReg := xbpReg():NEW("HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkStation\Parameters") 

it think Registry Entry are CASE sensetive. have a look into your Registry

you also can try HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters
EnableOpLockForceClose -> 1
UseUnlockBehind -> 1

[skiman]

Hi Roger,

No ADS is no option for me. I can't ask to all my customers to buy an ADS license. I'm convinced that this would solve the problems.

I'm waiting for Arctica since two years, which would be the solution we need. The move to SQL will give us tons of work, but that is something we can plan, and have control about it.

[skiman]

Hi Jimmy,

We don't sell hardware, and we have no control on the systems our customers uses. We have a commercial application, and this should work on every Windows network.

So version of Windows, way the network is setup are things I can't control. I even don't know how this is done on our own network. This is also done by an external technicien.