April 24, 2024, 11:07:56 AM

News:

Own IWBasic 2.x ? -----> Get your free upgrade to 3.x now.........


File data changer

Started by Andy, November 28, 2015, 05:08:30 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Andy

November 28, 2015, 05:08:30 AM Last Edit: November 28, 2015, 05:37:33 AM by andy1966
Hi,

I was helping Brian with a problem, his data file consists of something like this:

"Surname","Firstname","Initials" etc.

Where the " symbol denotes the start of each data block and end of the data block.
Each data block is separated with a "," (comma).

Sometimes, he was getting double quote mistakes:
""Surname"".

So I wrote some code to 'strip' out the extra quotes.

I began to play around with this code and have written the attached program.

Data blocks read in can:
have up to 255 characters at the start of data (can also have nothing)
have up to 255 characters at the end of data (can also have nothing)
the 'separator' can be up to 255 characters.

Data blocks saved in a new file can:
have up to 255 characters at the start of data (can also have nothing)
have up to 255 characters at the end of data (can also have nothing)
the 'separator' can be up to 255 characters.

You can also remove an unwanted character
You can replace a string with something else (or leave the replacement empty if you do not want to replace it with anything).

I have attached a zip file which includes some screenshots.

Just thought it might come in handy for someone.

Anyway, you can have a bit of fun with this one.

Andy.
:)



Day after day, day after day, we struck nor breath nor motion, as idle as a painted ship upon a painted ocean.

Andy

Sorry, forgot to say, run the program and browse to open test.txt.
Day after day, day after day, we struck nor breath nor motion, as idle as a painted ship upon a painted ocean.

Brian

Andy

Nice try - I like it, but it does fall over when there is a character inside a block which matches
what you are using as a field delimiter

Eg, if the comma is the delimiter, and a field is something like "Olive Tree, Olive Mount" not
removing the quote inside the field will result in a csv program reading the comma as an extra
field. That's the beauty of having each field surrounded by quotes - if there is a character inside
the field which matches the delimiter, if the field is quoted, then that character is ignored

Probably needs the comma stripping out (or whatever character is the delimiter)

Brian

Andy

Hi Brian,
Thanks for that, never thought to check for that situation.

I will have a look at that tomorrow.

Andy
:)
Day after day, day after day, we struck nor breath nor motion, as idle as a painted ship upon a painted ocean.

billhsln

This looks like it was a standard Comma Quote delimited file.  Take a look at a program I wrote years ago that will correctly handle a file in this format.

Bill
When all else fails, get a bigger hammer.

Andy

Thanks Brian and Bill.

Spent the remaining weekend re-writting the code.

This should now check and knock out wrongly inserted delimiters insided blocks of data.

Can someone try it for me please.

Have a look at "NewTestFile.txt" first to see the errors in the first few lines.

Compile, run and browse to "NewTestFile.txt" to open it.

Select an output file and wait for the confirmation window to appear.

Click ok to end.

Now have a look at your output file.

See attached.

Bill, I now have time to look at your code - thanks for that.

Thanks,
Andy.
:)
Day after day, day after day, we struck nor breath nor motion, as idle as a painted ship upon a painted ocean.

billhsln

Just so you know, I was parsing a standard comma quote delimited file which had Washington DC address's in it. They have weird address's with things like Avenue "A" being a valid address, which shows up like ,"13 Avenue """A"", which comes out as 13 Avenue "A", which is the correct style address for them.

Bill
When all else fails, get a bigger hammer.

Andy

Thanks Bill,

That's great to know!

Also, I like the code you posted here.

Thanks,
Andy.
:)
Day after day, day after day, we struck nor breath nor motion, as idle as a painted ship upon a painted ocean.