Problem with DC_Xml2ObjectTree

[Piotr D]

H, Roger,
I tried read an XML file into arrays. I use your DC_Xml2ObjectTree() function, these wok good with files with about 20 MB
(ca 400 000 lines). But when I use these function with bigger XML file (ca 40 MB, more than 800 000 lines), then program
terminated with XPPFATAL.LOG with error code 1006 and XPP:15. Computer was with Win7, 16GB RAM. Is it true, that
so big XML file can't be translated to object tree?

Piotr

[Auge_Ohr]

... XPPFATAL.LOG with error code 1006 and XPP:15.

1006 two many memory-objects: there are not enough handles.
15 XPP_ERR_MEMORY_FULL Not enough memory or swapping space available

try to increase STACK ... or ask Alaska for Patch to get more Handles.

[rdonnay]

I was not aware of this.

Please send me your XML file.

I don't know if it is a problem with the XML parser or the DC_XmlNode() class.

[Piotr D]

Roger,
Monday I send you via e-mail these XML file, XPPFATAL.LOG and source code.

Piotr

[Piotr D]

Roger,
attached is XML file, error log and source code.
I was compiled with Xbase 2.0 with Express 264

Piotr

[Victorio]

Hi,

I do not know function DC_Xml..., but I had similar memory problem with reading large text files, read file about 400000MB to string was not problem, but when read bigger, then memory error if I want some string operations or read part of file to array.
Because I must break reading large files to parts and process separately.

[rdonnay]

Piotr -

In my opinion, you are using XML for a purpose it was not intended, even though it does not invalidate any XML specifications.

Your XML file is 44,233,975 bytes. It includes 855,798 nodes.
Using DC_Xml2ObjectTree() would require building 855,798 separate DC_XmlNode() objects in a single tree.

This is totally impractical for many reasons.

It appears that you have converted a database of about 53,430 records to an XML document.

What do you intend to do with this data?
For what purpose?

Xbase++ must allocate far too much memory to be able to handle this. It locked up my sample program.

When I tried to load the XML into Internet Explorer it also locked up Internet Explorer.

Apparently, you are trying to find a mechanism to send a database via the internet.
Rather than converting to XML, it is probably best that you use Base64 encoding to attach the entire database to an XML document.

[rdonnay]

Here is an example of embedding a database into an XML document.

Here are the steps:

1. Load the database contents into a binary string.
2. Convert the string to a Base64 encoded string.
3. Build the XML with the encoded string.
4. Write the XML to a file.
5. Load the XML into an object tree using DC_Xml2ObjectTree()
6. Find the node containing the encoded data.
7. Convert the BASE64 encoded data back to a binary string.
8. Write the binary string to a new database.
9. Open the new database.

Code: Select all

#INCLUDE "dcdialog.CH"
#INCLUDE "fileio.CH"

#Pragma Library("dcxml.lib")

#DEFINE CRLF Chr(13)+Chr(10)

FUNCTION Main()

LOCAL cString, nHandle, nSize, cEncodedString, cXml, oRoot, ;
      oDataNode, cDatabaseName, oNode

nHandle := FOpen('books.dbf')

nSize := FSeek( nHandle, 0, FS_END )

cString := Space(nSize)
FSeek( nHandle, 0, FS_SET )
FRead( nHandle, @cString, nSize )

cEncodedString := Bin2Base64(cString)

FClose( nHandle )

nHandle := FCreate('BOOKS.XML')

cXml := '<?xml version="1.0" encoding="UTF-8"?>' + CRLF
cXml += '<Database Name="Books2.dbf">' + CRLF
cXml += '  <MimeType>application/dbf</MimeType>'
cXml += '  <EncodedData>' + CRLF
cXml += cEncodedString + CRLF
cXml += '  </EncodedData>' + CRLF
cXml += '</Database>'

Fwrite( nHandle, cXml )
FClose( nHandle )

oRoot := DC_Xml2ObjectTree('BOOKS.XML')

oNode := oRoot:findNode('Database',,.t.)
cDatabaseName := oNode:getAttr('Name')

oDataNode := oRoot:findNode('EncodedData',,.t.)

cEncodedString := oDataNode:content
cString := Base642Bin(cEncodedString)

nHandle := FCreate(cDatabaseName)
FWrite( nHandle, cString )
FClose(nHandle)

USE (cDatabaseName)

DC_Dot()

RETURN nil

* ---------

PROC appsys ; RETURN

[Auge_Ohr]

hi

2. Convert the string to a Base64 encoded string.
...
7. Convert the BASE64 encoded data back to a binary string.

Code: Select all

#INCLUDE "dcdialog.CH"
#INCLUDE "fileio.CH"
#Pragma Library("dcxml.lib")
...
cEncodedString := Bin2Base64(cString)
...
cString := Base642Bin(cEncodedString)

where do i find both function ?

[rdonnay]

where do i find both function ?

These functions are part of the Foundation Subscription of Xbase++ 2.0.

They are in the XPPRT1.DLL.