A more elegant way to use ADS SQL

This forum is for eXpress++ general support.
Message
Author
User avatar
rdonnay
Site Admin
Posts: 4734
Joined: Wed Jan 27, 2010 6:58 pm
Location: Boise, Idaho USA
Contact:

A more elegant way to use ADS SQL

#1 Post by rdonnay »

Added new SQL command and new DC_AdsSqlStatement() function. This provides a functionality equal to the SQL SELECT feature of Xbase++ 2.0, however it only works with Advantage Server and a Data Dictionary. Look at the sample program in \exp19\samples\adssql\SqlTest.prg. The command resides in DCADS.CH. The SQL command will return the result set in a workarea, in an array, or in an array of objects or all of them.

After working with the new SQL SELECT statement in Xbase++ 2.0, I became frustrated because it always failed with an Ads DacSession(). Apparently, Alaska has given this low priority over other SQL priorities. I also am disappointed with the performance of PostGreSql as compared to ADS. As an ISAM database it is unusable on large datasets.

After looking at the features of the Xbase++ 2.0 SQL SELECT command, I realized how easy it is to give this same functionality in Xbase++ 1.9 when using ADSDBE.

I have attached 4 files:
DCADS.CH - Contains the new SQL command
_DCADS.PRG - Contains the source for DC_AdsSqlStatement()
SQLTEST.PRG - A test program that demonstrates this new feature.
SQLTEST.XPJ

Copy DCADS.CH to your \exp19\include folder.
Copy _DCADS.PRG to your \exp19\source\dclipx folder and run BUILD19_SL1.BAT to rebuild DCLIPX.DLL.
Copy SQLTEST.PRG and SQLTEST.XPJ to your \exp19\samples\adssql folder.

Look at the source for how this new command can be used:

The single SQL command returns the result set in a workarea, an array (Scatter()) and an array of objects (DC_DbScatter()).
The 3 browses show how this data is used.

There is a DacSession() that connects to the Samples.Add (dictionary) in \exp19\samples\adssql\files\data folder.

Here is the syntax of the command:

Code: Select all

 SQL <statement> ;
   [INTO ARRAY <aData>] ;
   [INTO OBJECTS <aObjects>] ;
   [INTO CURSOR <cAlias>] ;
   [VIA <oSession>] ;
   [EVAL <bEval>] ;

Code: Select all

#INCLUDE "adsdbe.CH"
#INCLUDE "dcads.CH"
#INCLUDE "dcdialog.CH"
#INCLUDE "dmlb.CH"

FUNCTION Main()

LOCAL GetList[0], oTab1, oTab2, oTab3, oBrowse1, oBrowse2, oBrowse3, ;
      aInvoices[0], aObjects[0], cAlias := 'INVOICES', i, aStru, cSql

TEXT INTO cSql WRAP
SELECT
Invoice.Inv_nmbr as Invoice,
Invoice.Balance as Balance,
Customer.Bill_name as Customer,
Customer.Phone as Phone
FROM Invoice
LEFT OUTER JOIN Customer ON invoice.cust_nmbr = Customer.cust_nmbr
WHERE Invoice.Balance > 0
ENDTEXT

SQL cSql INTO ARRAY aInvoices INTO OBJECTS aObjects INTO CURSOR cAlias ;
    EVAL {|a|ATail(aInvoices)[3] := Upper(a[3])}

aStru := INVOICES->(dbStruct())

@ 0,0 DCTABPAGE oTab1 SIZE 130,25 CAPTION 'Work Area'
@ 0,0 DCTABPAGE oTab2 RELATIVE oTab1 CAPTION 'Array'
@ 0,0 DCTABPAGE oTab3 RELATIVE oTab2 CAPTION 'Objects'

@ 2,2 DCBROWSE oBrowse1 PARENT oTab1 SIZE 126,22 ALIAS cAlias
FOR i := 1 TO Len(aStru)
  DCBROWSECOL DATA DC_FieldWBlock(aStru[i,1],'INVOICES') PARENT oBrowse1 ;
    HEADER aStru[i,1] WIDTH aStru[i,3]
NEXT

@ 2,2 DCBROWSE oBrowse2 PARENT oTab2 SIZE 126,22 DATA aInvoices
FOR i := 1 TO Len(aStru)
  DCBROWSECOL ELEMENT i HEADER aStru[i,1] WIDTH aStru[i,3] PARENT oBrowse2
NEXT

@ 2,2 DCBROWSE oBrowse3 PARENT oTab3 SIZE 126,22 DATA aObjects
FOR i := 1 TO Len(aStru)
  DCBROWSECOL OBJECTVAR (aStru[i,1]) HEADER aStru[i,1] WIDTH aStru[i,3] PARENT oBrowse3
NEXT

DCREAD GUI FIT TITLE 'Browsing a SQL cursor'

RETURN nil

* -----------

PROC appsys ; RETURN
Attachments
sqltest.zip
(31.8 KiB) Downloaded 739 times
The eXpress train is coming - and it has more cars.

c-tec
Posts: 379
Joined: Tue Apr 20, 2010 1:36 am
Location: SALZBURG/AUSTRIA
Contact:

Re: A more elegant way to use ADS SQL

#2 Post by c-tec »

Hello Roger,
this looks really great, I will beginning next year try to convert a part of my software to ADS, this will make it much easier. I have also bad experience with SQL and speed, especially whe browsing large datasets. But ADS seems to be much faster.
regards
Rudolf
Rudolf Reinthaler
digital pen & paper systems
http://www.formcommander.net

Cliff Wiernik
Posts: 605
Joined: Thu Jan 28, 2010 9:11 pm
Location: Steven Point, Wisconsin USA
Contact:

Re: A more elegant way to use ADS SQL

#3 Post by Cliff Wiernik »

We currently do not use SQL on ADS, just natively via the ADSDBE. ADS is stable and fast. We have extremely hard record sizes and number of records. Some files exceed 4GB in size. No problems.

patito
Posts: 121
Joined: Tue Aug 31, 2010 9:01 pm

Re: A more elegant way to use ADS SQL

#4 Post by patito »

Hi Rudolf

In this matter definitely native classes are the solution. By using native classes for MySql and Postgres you will have a fastest solution,
cause you will gain speed and performance.
Is very important that you have in mind the difference between DBF Tables and SQL Databases. You must consider facts like building
Sql Tables using relation principles, among others.
Do you remember the ACID rules ?

Regarding the "browse" implementation, I think the best way to deploy a fast browser is by using pagination techniques.
Its always possible that your SQL SELECT statement query may result into thousand of records. But its is not good idea to display
all the results on one page. So we can divide this result into many pages as per requirement.
Paging means showing your query result in multiple pages instead of just put them all in one long page.
MySQL helps to generate paging by using LIMIT clause which will take two arguments. First argument as OFFSET
and second argument how many records should be returned from the database. (Idem Postgresql)
I've uploaded two free classes, one for MySql and the other one for PostgreSql.
Several programmers have been contributed with great ideas and important updates to this libraries.
Other users have built their own libraries using only the wrapper and reached speeds to 500%

Best Regard

Héctor

User avatar
sdenjupol148
Posts: 151
Joined: Thu Jan 28, 2010 10:27 am
Location: NYC

Re: A more elegant way to use ADS SQL

#5 Post by sdenjupol148 »

Hi Everyone,

I just want to add my 2 cents here.
We don't use SQL databases but we do use SQL and ISAM statements with ADS.
I currently do not see an advantage of SQL databases over ADS.

We have databases well over 20 gigabytes in size with millions of records that we can access instantly through browsing without the need to paginate.
Clients can search within the browses and if data subsets are needed, scoping allows us to show them instantly.
I have not yet seen SQL perform at this level and speed.

If, in the future, Postgres gives me this type of speed, flexibility and stability, I may move to it, but as of now, I recommend staying put with ADS to all my clients.

In addition, if having a fast Postgres database is dependent on developers writing their own ISAM wrapper, then I do not see the point of using xBase++ with Postgres.
Imagine purchasing a new car that doesn't go faster than 20 miles an hour but the salesman tells you can modify the engine yourself to make it go faster.
To me, this is a flawed product; I would rather stay with my old car that can go 100 miles an hour.
The idea of paying money for a product is that it should work well right out of the box.
That's what I currently get with ADS; a product that works flawlessly, is incredibly fast and maintains excellent support.

Just my 2 cents

Bobby

Cliff Wiernik
Posts: 605
Joined: Thu Jan 28, 2010 9:11 pm
Location: Steven Point, Wisconsin USA
Contact:

Re: A more elegant way to use ADS SQL

#6 Post by Cliff Wiernik »

I agree with Bobby. We have similar size files in terms of data file size and number of records, one with 78 million records. No speed issues. Our main database for browsing where users select accounts has 200,000 records and does incremental searching and is absolutely responsive with no delays. It display 30 records at a time in the browse. As each of the characters of the 9 digit account number is entered, the new page of data displays. It uses coding from Express so that if you key in digits quickly, it does not necessarily search after each key.

We also have a collection comment screen, the 78 million record file. It is a browse on a tabpage. When we navigate from one the account to the next, or retrieve an account, the display of the current accounts collection comments is displayed basically immediately. A customer may have 1 page of comments or 100 pages, no delays in either case.

It would appear that PostGres has a long way to go to meet the performance provided by ADS. ADS does have a cost, but it is relatively cheap for what you get. Microsoft's office products are much more expensive in comparison, outright and on an ongoing basis.

Cliff

User avatar
rdonnay
Site Admin
Posts: 4734
Joined: Wed Jan 27, 2010 6:58 pm
Location: Boise, Idaho USA
Contact:

Re: A more elegant way to use ADS SQL

#7 Post by rdonnay »

Just my 2 cents
Bobby -

O' the irony of your last sentence.
Advice has a value. You put yours at 2 cents. If it was only worth 2 cents you wouldn't have given it.
To me, your kind of advice is invaluable, because you are advising out of real concern for your clients and our Xbase++ community.

Occasionally, I make a posting on "The Skeptics Guide to the Universe" forum.

I posted the following about giving advice, when a podcast listener chastised the host for not giving his opinion about a moral issue to one of his students:
When young people are given facts and evidence that are true, they tend to develop a less convoluted world view and thus tend to make more ethical choices and decisions. When they are given opinions, it only adds to their confusion. There are thousands of opinions, and they all contradict each other. Facts and evidence have the opposite affect. They tend to filter out the noise and help young minds to approach life with a more skeptical attitude and a greater ability to develop their own opinions.
Software developers, and their clients, are always confused about which way to turn. So they come to public forums like this with the hope that it will all be made clear to them. Instead, they walk away more confused than ever. Why? Because they have not developed a "trust" relationship with those who are advising them. Trust comes from knowing that your adviser speaks truth, facts and opinions that are in your interest. Trust comes from knowing that your adviser understands your problems, your history, your application, your skills, your limitations, and your fears. Advice must be given in a proper context, but often it is not. Most often, advice is given to serve the adviser, not the client. Good advice is never free, and good software is never free, yet many programmers still tend to think that free software and free advice is the best policy.

I don't advise all programmers that they should use Advantage Server, or even eXpress++. I have several Xbase++ users who are my clients who don't even know about eXpress++. I have determined that it is not in their interest to know about it because it would only confuse them. Like Bobby, I get frustrated when other programmers make postings that are self-serving. Every product mentioned in this thread and other threads has value, but not value to everyone. Xbase++ has immense value to all of us, but no value to a Java programmer. SQL databases are languages. The SQL database that is the most like Xbase++ is Advantage Server. That is the truth. What this means is that Advantage Server is usually the best choice for large Xbase++ and Clipper applications with lots of large databases when the requirement is to upgrade to client/server and/or SQL.

Let me be clear. I do not work for Sybase nor do I get anything from Sybase when Advantage Server is sold. I am not a reseller and I don't have any Sybase addon products. My advice comes from over 25 years of working with Advantage Server and successfully helping dozens of customers make the migration with never a failure or a performance disappointment.

My 200 dollars.
The eXpress train is coming - and it has more cars.

patito
Posts: 121
Joined: Tue Aug 31, 2010 9:01 pm

Re: A more elegant way to use ADS SQL

#8 Post by patito »

Hi Roger

Thanks for your feedback
Hi Bobby If our customers are willing to pay over 4000 dollars to buy ADS, I'm with you.
The topic when speaking of using Postgres or Mysql with alaska, is migrate it like a table dbf


Best Regard
Héctor

User avatar
rdonnay
Site Admin
Posts: 4734
Joined: Wed Jan 27, 2010 6:58 pm
Location: Boise, Idaho USA
Contact:

Re: A more elegant way to use ADS SQL

#9 Post by rdonnay »

If our customers are willing to pay over 4000 dollars to buy ADS, I'm with you.
Hector -

That has always been an easy sell for me.

Most of the customer whom I have advised an ADS purchase are multi-million, or billion dollar businesses.
However, even smaller businesses get a return on an ADS investment.

For example, I recall many customers who had to shut down and reindex every night because an improper workstation shutdown would corrupt index files. This had a cost associated with it. Let's say that cost was $10 per workday. An average of 300 workdays per year means a cost of $3000 per year. That is a very low estimate. The cost to update the code was maybe $1000 of programming time. Using any other means to convert to client/server would cost tens or hundreds of thousands of dollars because all the code would need to be re-written.

Cost justification is the main reason to use ADS.

Roger
The eXpress train is coming - and it has more cars.

c-tec
Posts: 379
Joined: Tue Apr 20, 2010 1:36 am
Location: SALZBURG/AUSTRIA
Contact:

Re: A more elegant way to use ADS SQL

#10 Post by c-tec »

Hello,
I have the same problem as Bobby, I have a lot of code that uses several tabpages and browsers with incremental surch functions. With DBF files my customers love the way to tip only some letters and see immediatly all records around the target, and this with hundredthousands of records. So for example he can see all customers with similar names and can pickupt the correct one or check if it exists. And if the user click to ohter tabs with invoices, addresses etc. the data is also immediatly available. I see way to do this SQL statements, datasets and cursors.
I see a lot of advantages in using SQL, but it is hard for me to change my code to an much slower and not so comfortable solution where the customer cannot see the benefit. I would also have to change a lot of code to resovle the speed penalty. In the past I had a big project with SQL and made my own logic for fetching site by site, but it was too complicated and too slow. For now I use SQL only for replicating DBF tables for customized interfaces and access to other software.
regards
Rudolf
Rudolf Reinthaler
digital pen & paper systems
http://www.formcommander.net

Post Reply