Translate scripts into commands on server

[Mike Doty]

Got it, thank you!

[John McWilliams]

"client starts listening for the results to come back"
How does a disconnected client listen for the results?

They open a listening port and wait (FTP server?, WEB server? come on this is net 101)

How does a server contact a disconnected client?

You have their name, IP address, MAC Address and their authentication (which they have saved for that session and will respond with), so simply contact them on the random port that they have provided you and probe their MAC, IP Address and Name, then ask for their authentication. If that checks give them the data over the port they requested. Then disconnect.
If you have no persistent connections then you have no "Man in the middle" attacks. If you have no persistent authentication then you have no spoofed sessions.

Is the server opening databases in exclusive mode?

The server is opening the connection, it's always exclusive (for a whole 500 ms)

So each client opens up a port as a server for responses?

Exactly. It's a stateless connection. Open-request-close-listen-get-act

[Mike Doty]

"client starts listening for the results to come back"
How does a disconnected client listen for the results?

How does a server contact a disconnected client?

Is the server opening databases in exclusive mode?

So each client opens up a port as a server for responses?

[John McWilliams]

John,
Do you open files on the server for exclusive access or does each
new connection simply wait in line? Do you use any threading?

Sorry have been away for a while. Let me explain the procedure.

client connects
client passes request to service
client disconnects (this takes a whopping 20ms)
client starts listening for the results to come back
server gets request
server starts worker thread
server gathers data
server formats data
server contacts client
server sends data
server disconnects (WOAH! a whole 500ms! depending on data size)

Thread works until it either times out or has the data. If the thread times out it sets the RETURN Value: "NO_DATA:TIMEOUT". If the tread succeeds then it sets the Return Value: "SUCCESS:<DATA STREAM>. Server contacts client and gives client the return value. Now it's up to the client to do something with it.

No client is connected to the server for as long as it takes to pass a request or receive a return message/data stream. THERE is *no* long term connection. It's a statefull connection. The states are simple.

"I need data" - Connect me!
"here's what I need" - disconnect me
"I have the data" - connect to the WS
"Here's the data" - here it comes
"I'm done with you" - disconnect.

I can serve literally 100,000's of thousands of connections with that strategy. I can get bogged down with CPU usage in processing the requests but I simply queue the request when I get bogged down (but that's *VERY* rare)

[Mike Doty]

John,
Do you open files on the server for exclusive access or does each
new connection simply wait in line? Do you use any threading?

[Michael Mattias]

SOAP and XML/RPC are EDI... electronic document interchange.

Definition of EDI:

EDI Overview

Definition:

EDI (Electronic Data Interchange) is the one-way transmission and reception of common business documents in computer-system-readable format. Documents are formatted using either public or private standards.

The most common public standard used in The United States is the ANSI ASC X12 standard.

The UN EDIFACT standard is common in Europe; the TRADACOMS standard is used primarily for financial transactions.

An example of a private standard is the ACH standard used (only) for funds transfers.

Usage

EDI is used primarily to transmit documents which are typically printed on pre-printed forms: Invoices, Purchase Orders, Shipping Manifests, Remittance Advice. ANSI and UN/EDIFACT have defined standards for many documents; see the attached list of document types supported by the ANSI X.12 standard.

Many documents are industry-specific: e.g., Mortgage Escrow Billing, Health Care Claim, Property/Casualty Loss Notification.
...

Source: "EDI For Everyone", copyright 1996, 1998, 2006 Michael C. Mattias Racine WI USA

MCM

[Brian Chirgwin]

No, No, and No.

Both ANSI ASC X12 and EDIFACT EDI standards are used every day, many times each day.

No, it never evolved into what many of us in the early 1980s dreamt it could be, but it is far, far from dead.

Some of us (guess who!) have put a roof over our heads and food on our tables for twenty-five-plus year performing IT services and developing IT products almost exclusively dedicated to the support of ANSI ASC X12 and EDIFACT EDI standards for business communications.

Check out the website below.

Michael Mattias
Tal Systems Inc.
Racine WI

http://www.talsystems.com

PS: We now do EDI over the Internet. The internet is merely the medium, not the message.

I didn't mean to indicate EDI was dead. I don't think I said this or left the impression. I definitely didn't mean to. If I left that impression; I am corrected.

I was only trying to indicate that SOAP and XML/RPC are two language independent protocols that have gained wide acceptance. EDI was suppose to be such a "standard" but didn't meet its marketing hype.

[Michael Mattias]

Before the internet there was EDI (for business to business communication) and is still used today, but it never gained wide acceptance as each vendor added additional special features which made them incompatible with other vendors. So Business A has Vendor A's product and Business B had Vendor B, they couldn't communicate because EDI was no longer a standard.

No, No, and No.

Both ANSI ASC X12 and EDIFACT EDI standards are used every day, many times each day.

No, it never evolved into what many of us in the early 1980s dreamt it could be, but it is far, far from dead.

Some of us (guess who!) have put a roof over our heads and food on our tables for twenty-five-plus year performing IT services and developing IT products almost exclusively dedicated to the support of ANSI ASC X12 and EDIFACT EDI standards for business communications.

Check out the website below.

Michael Mattias
Tal Systems Inc.
Racine WI

http://www.talsystems.com

PS: We now do EDI over the Internet. The internet is merely the medium, not the message.

[Brian Chirgwin]

SOAP or XML/RPC

Mike,

I think what you are getting at is a protocol that any language can support (some support them more than others) to execute a command. There are several of these developed over the years. The latest and most successful are SOAP and XML/RPC.

Before the internet there was EDI (for business to business communication) and is still used today, but it never gained wide acceptance as each vendor added additional special features which made them incompatible with other vendors. So Business A has Vendor A's product and Business B had Vendor B, they couldn't communicate because EDI was no longer a standard.

SOAP and XML/RPC are successful because they are a contract/definition of the functions and returned results. Similar to the way COM is a definition. The Contact can't be broken or it breaks the communication between client/server, but the standard isn't owned by a given vendor. The protocols (SOAP and XML/RPC) are generic, again like COM.

[John McWilliams]

Looks great. First answer to my question!
Looks like you wrote a parser that feeds the fields to the functions needed.

Yep!

As far as the packets, I haven't been doing any security at the packet level.

Shouldn't need to.

Curious, does your parser work within a single function to give you the results to send back?

Works like a message cracker

Have you had any problem sending or receiving packets greater than 1460 bytes? I may have a problem on this test machine.

I don't care what the packet size is. I let the OS deal with that. That's what it get's paid for. LOL.

Is the client logged off after every request?

Yes.

TimeToExecuteTheRequest: T+10 (Maximum of 10-seconds?)

I also use NOW. That pushes the request to the top of the list if others have more lenient needs. T+10 means I need my results in 10 minutes or less.... I'm listening.......

ReturnTheInfoTo: Curious why the IP address if the client is already connected?

Results are ONLY sent to that address and it better be an internal address! I need to qualify that. Every WS is required to authenticate on initial contact. There is a procedure that each client needs to go through before they can connect. Kind of a self registration, with restrictions. That address is then linked to a MAC address.... it gets complicated (network stuff) but it works great. No one's been able to hack it because it changes at each connection..... can we say moving target?

Authentication: Some method

Yep.... roll your own!

Greatly appreciated!

You're Welcomed

[Mike Doty]

Looks great. First answer to my question!
Looks like you wrote a parser that feeds the fields to the functions needed. As far as the packets, I haven't been doing any security at the packet level.
Curious, does your parser work within a single function to give you the results to send back?
Have you had any problem sending or receiving packets greater than 1460 bytes? I may have a problem on this test machine.
Is the client logged off after every request?

MachineName:UserName:Request:TimeToExecuteTheReque st:IPAdressTo ReturnTheInfoTo:Authentication
Client01:JohnMcWilliams:SELECT ALL,RECORD 183-299,SORT LASTNAME:T+10:192.168.47.100:16262435679964523806

MachineName: Client01
UserName: User
Request: (answers my question)
TimeToExecuteTheRequest: T+10 (Maximum of 10-seconds?)
ReturnTheInfoTo: Curious why the IP address if the client is already connected?
Authentication: Some method

Greatly appreciated!

[John McWilliams]

John,
I'm not having any problems in transmittting or receving.
However, do you autheniticate every packet?

The question is on how you process the requests from the server like
give me client record 183 to 299 and sort by last name.

You're lost in the minutia! I don't authenticate any packets. I authenticate the Client. I could care less about the packet content, just what it translates to.

i.e. :

Client01:JohnMcWilliams:SELECT ALL,RECORD 183-299,SORT LASTNAME:T+10:192.168.47.100:16262435679964523806

[Mike Doty]

I've noticed that packets larger than 1460 bytes can come back incomplete. I may post some code on this later. Does someone
have an improved TCP RECV loop to prevent buffer over-run? Since setting my receive buffer to 1460 no errors have occurred.

[Mike Doty]

John,
I'm not having any problems in transmittting or receving.
However, do you autheniticate every packet?

The question is on how you process the requests from the server like
give me client record 183 to 299 and sort by last name.

[John McWilliams]

Way to Complicated

Mike you are getting bogged down in the minutia. Just listen on a port and wait for a formatted command to come in and then execute it (security hole?). Who cares what language sent it? I've been doing stuff like that for years. I use an authentication procedure to verify who sent the request (and I journal it) before I act on it. But I just listen on a port and as long as the "message" fits the format and the client is authenticated then it gets done. I do something like this:

MachineName:UserName:Request:TimeToExecuteTheRequest:IPAdressTo ReturnTheInfoTo:Authentication

If the request doesn't meet the format or doesn't authenticate, I ignore it.

[Michael Mattias]

The client is in PowerBASIC, but the client could be in another language if it knows the format of the send/receive buffers.
Only the server is allowed any access to the data files (which you all know.)

???

The source language of the client program should not be material at all. If that client can send a command to the server - by TCP or any other method you choose to support - the server can respond.

It sounds to me like you currently have a very workable server solution.

> think in datasets which is fine.

RDS (remote data services) ??

[Mike Doty]

Yes, that is correct.
The client is in PowerBASIC, but the client could be in another language if it knows the format of the send/receive buffers.
Only the server is allowed any access to the data files (which you all know.)

I haven't seen any postings on how others layout their send and receive buffers or parse out commands and process them. I was trying
to come up with a better way. I'm going to study some web server code and see if I can adapt that. Currently, I just makeup commands.
There is nothing wrong with this except I would rather have a standardized set of commands. I see some use a 2-dimensional array and
think in datasets which is fine. It would also eliminate the need to keep record position and current key number for each connection if access is only
allowed within a transaction. The first access would establish or reestablish the record and key position for each connection.

[Cliff Nichols]

I think I get where Mike Doty it going with this.

Do you mean, that you write some client/server program, but allow me to write a text script that could do something like

For i = 1 to 5
     Open Connection Yadda(i)
next i

And I not care what language you wrote the program in? nor you caring what language (or script) I am calling it from????

[Michael Mattias]

>Instead of the code below

This is server-side code, right?

Why replace it? It looks pretty good from here.

Or do you mean something like, '"PUT" or "GET" imply using a file previously opened with an "OPEN" command (buffer="OPEN")???

You can do that, you just have to keep track of who has what file open and where the file pointer is.

{ADDED}

Maybe you can show a little suggested "client-side" pseudo-code? That is, what you are desirous of supporting?

[Mike Doty]

Instead of the code below a predefined command set or parsing method.

Roll your own:'
TCP RECV [I]fNum&[/I], [I]count&[/I], [I]Buffer$[/I]
IF Buffer = "GET" THEN 
  CALL GETROUTINE parameters
ELSEIF Buffer = "PUT" THEN
  CALL PUTROUTINE parameters
ELSE 
END IF