PDA

View Full Version : importing Tap .ml into Evolution


dgermann
July 6th, 2006, 08:31 PM
Hi--

Anybody know what is necessary to import tapcis mailing list files into Evolution?

The Evo help screen says "Currently VCF and LDIF are supported. CSV and TAB format files are supported from Microsoft Outlook, Thunderbird, Mozilla, and Evolution." --if that is any clue to the duly informed.

Thanks for any help you can give! :D

davidh
July 6th, 2006, 09:55 PM
Hi--

Anybody know what is necessary to import tapcis mailing list files into Evolution?

The Evo help screen says "Currently VCF and LDIF are supported. CSV and TAB format files are supported from Microsoft Outlook, Thunderbird, Mozilla, and Evolution." --if that is any clue to the duly informed.

Thanks for any help you can give! :D

I can't give any specific help, but if a program allows importing comma (CSV) and tab delimited files, frequently it's sufficient just to put the desired column headings into the first row of the CSV file. You need to read the documentation of the destination (importing) program to find out which field names (field titles) that it wants to see. If you already have a CSV or TAB file you can plug in the column (field) titles (headings) by editng the CSV or TAB file with plain text editor such as Notepad or Wordpad in Windows or whatever text editor you have on your Operating System.

Open Office handles CSV files or TAB files quite nicely (as does MS Excel too). In case you don't have MS Excel or don't want to use it for whatever reason.

DH

davidh
July 6th, 2006, 09:58 PM
Of course, if you're using a LINUX or UNIX text editor you'll have to watch out for the line terminations (endings). Windows uses CR LF and UNIX usually uses LF (linefeed) only. Wordpad in Windows will probably edit either kind of file (Windows or UNIX text files) ok more or less.

DH

Lindsey
July 6th, 2006, 10:10 PM
Don't forget that you will need to change any CompuServe addresses to Internet format.

--Lindsey

dgermann
July 7th, 2006, 05:14 PM
David--

Ahh! I wondered what CSV meant--gather it means Comma Separated Value.

You have given me some clues about what to look for, and I will.

Lindsey--

Very good point. I think some of my mailing lists are OK, but others still have a mixture of the old CI$ formats. Thanks.

Thanks to both of you, David and Lindsey!

Lindsey
July 7th, 2006, 05:38 PM
Good luck! Let us know if you need any further help. Transferring mailing lists from one application to another can be a giant pain, especially when one of the lists is not in a standard format and has to be edited. :-( But at least the TAPCIS mailing lists are readily editable.

You know, come to think of it, Pete Hall had a QBASIC program in the old TAPCIS library that might get you part of the way there. I don't have a copy of it on my system, but someone around here might have it.

--Lindsey

dgermann
July 8th, 2006, 05:16 PM
Lindsey--

Thanks! Hope someone has that program around.

I did some playing with it and am not done. I had tried importing something in to the program and it seemed to stall out. Since I could find nothing about what format they wanted for an import (as David suggested) I decided to try an export to see if that showed what they wanted, and the only thing it had was vcard format. So I exported what I thought was one single dummy address I had just entered.

Instead, it has a whole bunch of my addresses in there. The notes that come after the "|" in the .ml file are not there, but names and e-mails are. Not sure how many or how complete it is. It is possible that I had some old version of evolution up many moons ago on this machine and had figured out how to put in names. Hard to tell.

So some experimenting lies ahead. Maybe I will just delete this file (if I can figure out where it is) and put the names and addresses in by hand--that might be easier, even though there are probably 500+. <sigh>

Judy G. Russell
July 8th, 2006, 07:07 PM
Doug, if you don't mind my taking a look at the addresses, shoot me a copy of the file at jgr (at) tapcis.com (replacing the (at) with the at symbol, of course) and let me see if there's an easy way to do this...

davidh
July 9th, 2006, 04:16 PM
Lindsey--

Thanks! Hope someone has that program around.

I did some playing with it and am not done. I had tried importing something in to the program and it seemed to stall out. Since I could find nothing about what format they wanted for an import (as David suggested) I decided to try an export to see if that showed what they wanted, and the only thing it had was vcard format. So I exported what I thought was one single dummy address I had just entered.

Instead, it has a whole bunch of my addresses in there. The notes that come after the "|" in the .ml file are not there, but names and e-mails are. Not sure how many or how complete it is. It is possible that I had some old version of evolution up many moons ago on this machine and had figured out how to put in names. Hard to tell.

So some experimenting lies ahead. Maybe I will just delete this file (if I can figure out where it is) and put the names and addresses in by hand--that might be easier, even though there are probably 500+. <sigh>

Doug,
I downloaded the Novell Evolution 2.6 user guide from somewhere.

It mentioned importing CSV files from Mozilla. I have both very old Mozilla 1.1 *and* Mozilla Thunderbird installed on this PC, even tho' only my wife uses them, not me.

I did a test export of the address book from *both* Mozilla 1.1 and from the latest version of Mozilla Thunderbird.

The formats exported were VERY different. So this could be a problem.

The format exported by Thunderbird was simpler in a way:

I will number the columns / fields from A to E.

A = first name
B = surname
C = nickname and/or full name completely written out
D = this field was empty in all the entries I had to look at, so I don't know what it actually is
E = email address with ID, "@", and POP3 server domain, i.e. normal Internet email address of correspondent

In the Thunderbird exported address book, there were no column (field) headings (titles, field names, etc.). That is, the meaning or interpretation of the column or field was implied by its postion (A thru E).

Thunderbird, as I said is the somewhat simpler case. The CSV format address book exported by Mozilla 1.1 was more complicated (I won't go into details), suffice it to say, the CSV format exported by Mozilla 1.1 was *much* less compatible with TAPCIS ML file format than Mozilla Thunderbird CSV export file format.

I did notice that Mozilla 1.1 had an option for *importing* address books in CSV format. Therefore, assuming that the address book format of Mozilla email address books has remained consistent from version 1.1 up until the current version 1.7.12, you might be able to install Mozilla 1.7.12 to import a "fixed" edition of Tapcis ML file (as a CSV) and *then* export the address book (mailing list) as a Mozilla CSV, TAB, TXT whatever file for subsequent import into Evolution.

Since you have 500 addresses, this roundabout method might still be worthwhile looking into?

First issue to settle definitely is whether Evolution imports Mozilla address books or Thunderbird address books in CSV format (or both or neither).

Second (or maybe first) issue to settle is how hard is it to convert TAPCIS ML files into CSV format. Using a text editor with global search and replace capability this should not be extremely difficult (but also not trivial).

One thing to watch out for is whether the pre-existing TAPCIS ML file has entries with commas. If so, in such a situation, I myself would often consider replacing such commas with alternate punctuation such as semicolon. This can be a little tricky when using a text editor. One may have to complete the conversion in more than one stage (e.g. multiple global replacements of punctuation marks). Hint: vertical bar (also call pipe) is a punctuation mark for the purposes of this discussion.

Here is a sample of a single entry in a CSV export file from address book of Mozilla 1.1, just to give you an idea of what's involved.

dn: cn=nicknameofsomebody,mail=idofsomebody@netscape.n et
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
objectclass: mozillaAbPersonObsolete
cn: nicknameofsomebody
mail: idofsomebody@netscape.net
modifytimestamp: 0Z

As you can see it is a plain text file, with *no* comma delimiters. No delimiters needed since each field is preceded by an exact textual specification of the field name/purpose.

(BTW, this type of format is also a sort of default import format for Pegasus email client address books.)

I used to have Outlook 98 free edition installed on my PC long ago, but no longer, so I don't know what it's export address book format looks like.

Perhaps a forum member who is more familiar with Mozilla can confirm how consistent the CSV format addressbook exports from Mozilla (not Thunderbird) are from version 1.1 to version 1.7.12 of Mozilla, to help you?

Caveat emptor,

DH

Gary Maltzen
July 10th, 2006, 12:18 PM
Here is a sample of a single entry in a CSV export file from address book of Mozilla 1.1, just to give you an idea of what's involved.

dn: cn=nicknameofsomebody,mail=idofsomebody@netscape.n et
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
objectclass: mozillaAbPersonObsolete
cn: nicknameofsomebody
mail: idofsomebody@netscape.net
modifytimestamp: 0Z
FYI - that is *not* CSV, but LDIF (http://publib.boulder.ibm.com/infocenter/tssfsv21/topic/com.ibm.sanfs22.doc/fog0_r_ldif_file.html) (LDAP Interchange Format)

CSV looks like this:id,first,last,addr
1,George,Washington,"White House"
2,Bill,Gates,Microsoft

davidh
July 10th, 2006, 02:25 PM
FYI - that is *not* CSV, but LDIF (http://publib.boulder.ibm.com/infocenter/tssfsv21/topic/com.ibm.sanfs22.doc/fog0_r_ldif_file.html) (LDAP Interchange Format)

CSV looks like this:id,first,last,addr
1,George,Washington,"White House"
2,Bill,Gates,Microsoft

Thanks for the correction. I must have clicked too quick. I tried exporting my address book from Mozilla 1.1 *again* and the file turned out the same format as the CSV file that I had exported from Mozilla Thunderbird 1.5.0.2.

I actually went back and opened up the Personal Address Book in Mozilla and selected one of the people (contacts) in the list and then clicked the Properties button to review/edit the contact card. The order of the fields to be entered in the dialog box that appears is

1. first name
2. last name
3. display name
4. nickname
5. email address

#3 and #5 appear to be the only essential entries. The others are apparently optional in Mozilla and Mozilla Thunderbird.

I also noticed 33 additional commas following each email address in the CSV export from Mozilla. I assume these are for additional optional fields that one can enter into the individual contact cards in the address book in Mozilla.

Also the field names do NOT appear in the CSV files exported from Mozilla. That is, instead of the first row/line of the CSV file being the names of the fields (something like "first, last, display, nick, email", for example), the first line of the file is actually the info for the first person in the list.

Based on this information, it seems that a CSV file to be imported into Evolution would have each line look something like this:

,,John Smith,,johnsmith@compuserve.com,,,,,,,,,,,,,,,,,,, ,,,,,,,,,,,,,,

Columns A, B, and D are empty , i.e. fields 1, 2, and 4 are empty.

Perhaps the 33 commas at the end are optional.

The notation field in the tapcis ml file might correspond to one of fields among those delimited by the 33 commas. One way to find out which field it is in Mozilla would be to install Mozilla (or Thunderbird) and fill in a address/contact card *completely* and then export the address book to a CSV file and see where each field entered falls in the 33 commas.

DH

dgermann
July 12th, 2006, 03:33 PM
Judy, David and Gary--

Wow! You folks are great!

Judy, I will e-mail you a shorter version of one of my mailing lists so you can get an idea what is there. Thanks for your generous offer.

How much work did you do on this David? Surely a bunch. I thank you very much.

David, when you said,
One way to find out which field it is in Mozilla would be to install Mozilla (or Thunderbird) and fill in a address/contact card *completely* and then export the address book to a CSV file and see where each field entered falls in the 33 commas.
That is approximately what I did in Evolution, getting the vcard format stuff.

I do have Pegasus on another machine--would it make any sense to do a two step importing into Pegasus and then out of there to Evolution?

I also need to take a look at the lists I have and see how much work they need. There may be a lot of deadwood there!

Thanks everyone for so much help. You are wonderful people, each and every one!

davidh
July 12th, 2006, 05:33 PM
Judy, David and Gary--

Wow! You folks are great!

Judy, I will e-mail you a shorter version of one of my mailing lists so you can get an idea what is there. Thanks for your generous offer.

How much work did you do on this David? Surely a bunch. I thank you very much.

David, when you said,

That is approximately what I did in Evolution, getting the vcard format stuff.

I do have Pegasus on another machine--would it make any sense to do a two step importing into Pegasus and then out of there to Evolution?

I also need to take a look at the lists I have and see how much work they need. There may be a lot of deadwood there!

Thanks everyone for so much help. You are wonderful people, each and every one! Doug,
I forget the format of tapcis address books or mailing lists.

However I vaguely remember that they have vertical bars (pipes) and probably <> brackets and/or double quotation marks.

If that is the case you should be able to edit the ml file with a text editor such as Wordpad and change the pipes, quotes, brackets into appropriate delimiting punctuation marks or combinations of punctuation marks. etc.

For example, change a pipe into a comma.

Or for example change a > bracket into a > bracket plus a comma such as >,

This can be done using global search and replace. Normally in the simpler cases you confirm each replacement by hitting a button or some key combination to indicate to the editor whether or not you really want the editor to do the replace for each particular instance in the whole (global) file. Depends on which editor you use and how many tricks you know for that editor.

Before replacing the existing punctuation, you probably will want to get rid of any commas that could cause confusion during the import to Evolution. For example, if you have set your "display names" in tapcis in the format "lastname, firstname", then you want the comma to stay there after the import and NOT be considered to be a field delimiter. One quick and dirty way to workaround this problem is to replace the comma with a semicolon. Sometimes, if the comma-containing field (item within a record) is surrounded with double quotes, the importing program will recognize that you do not want the field to be split. You will have to find this out by trial and error, most likely.

A good thing to do would be to use your text editor to make up a test file with comma delimited fields (in the Mozilla format) to see how Evolution reacts to it.

For example,

, , David Harper, , jdharper@compuserve.com

would be my old CIS email. Notice fields A, B, and D are empty.

, , "Harper, David", , jdharper@compuserve.com

is another example that may or may not work.

You will also need to figure out by trial and error (with Mozilla for example) how many commas to add after the email address and where in the bunch of commas to have the tapcis "notation" field sit. (you can do this with EXCEL, Open office, etc.) That is, your spreadsheet program can add columns anywhere you want, each time a column is added, appropriate delimiting commas are added to every record (contact/correspondent / row / line) in the file. Then exporting out of the spreadsheet program into a CSV file gives you the plain text CSV file you can eyeball with your editor.

If after you do the global replacements of old punctuation characters in the ML file with commas, the fields are still in the WRONG order or if there are TOO FEW fields, you can fix this by importing the ML file as a CSV file into EXCEL, Star Office, Open Office, Wordperfect, etc. and moving the columns around to get them in the correct order and/or adding columns to get the right number of columns. Then export the fixed file out as a new CSV file from EXCEL, etc. before the final import into Evolution.

Do some test runs with small sample files in whatever spreadsheet program you are using to get the hang of it and see what the CSV files look like.

The nice thing about CSV files is that they are just plain text files, so you can view them with a dumb vanilla editor or even a file dump program to see what you may or man not have changed during the stepwise reformatting of the file with text editor and EXCEL, Open Office, etc.

I would not recommend playing with Pegasus, since AFAIK it is not compatible with CSV or LDIF (this may have changed in recent versions).

I don't think you really would need to install Mozilla or Thunderbird either. Just make up a test CSV file with 2 or 3 records (your contacts from TAPCIS) in it according to the format stated in the other msg's here and see if Evolution vomits or what happens.

DH

Judy G. Russell
July 12th, 2006, 09:40 PM
Judy, I will e-mail you a shorter version of one of my mailing lists so you can get an idea what is there. Thanks for your generous offer.No promises, but I should have the tools to at least give this a try for you.

Lindsey
July 12th, 2006, 10:50 PM
I forget the format of tapcis address books or mailing lists.

However I vaguely remember that they have vertical bars (pipes) and probably <> brackets and/or double quotation marks.
Yes, but a couple of notes here:

The vertical bar was used to separate the address from the associated comments (and as I recall, there will be a vertical bar in the line even if no comments follow it). So you'd likely want to delete everything from the vertical bar to the end of the line. Example:

LOOFBOURROW-L@rootsweb.com |Loofbourrow mailing list (loofborrow)

Angle brackets may be used to denote an address in the standard way

<LOOFBOURROW-L@rootsweb.com> |

but it's more likely that a single right-pointing angle bracket will be used to "point" to the beginning of the actual address:

Buddy Bourne > INTERNET:xxxxxxxxx@erols.com |(Buddy)
Stuart Bourne >xxxxxxxxxxx@bellsouth.net |(Stu)

In which case you couldn't treat it like the right angle bracket that was part of a pair.

Before replacing the existing punctuation, you probably will want to get rid of any commas that could cause confusion during the import to Evolution. For example, if you have set your "display names" in tapcis in the format "lastname, firstname", then you want the comma to stay there after the import and NOT be considered to be a field delimiter.
TAPCIS itself did not have such an option that I can recall. There may, however, be names formatted that way in a mailing list/address book file, if that's the way it was saved from an incoming message. In that case, the name will be enclosed in double quotes.

--Lindsey

dgermann
July 14th, 2006, 03:32 PM
Judy, Lindsey, David--

Many thanks for all your help.

Last evening I took the file I had e-mailed Judy and edited it a bit. I took out all commas in the names and all periods (as in Ms.) and put in commas between the fields. Then I renamed the file test.csv.

Evolution took it and converted it, but the entries are not in the order Evo expected them. So I have more playing around to do.

It is slick to think of doing this with a spreadsheet rather than a text editor, David!

Lindsey you are right about what the .ml files look like.

As hush-hush as Judy is on this, it seems she must have some super-secret conversion program! ;)

davidh
July 14th, 2006, 03:51 PM
It is slick to think of doing this with a spreadsheet rather than a text editor, David!


Possible point of interest:

If you ever need to do this kind of thing from an Internet cafe (or at the library) when no spreadsheet software is installed locally, you can still do it online at http://spreadsheets.google.com

All you need is a free gmail account.

BTW gmail is pretty neat because you can read it with most regular POP3 email programs *and* on the web. And you get 2.7GB storage for email and you can forward your mail to a pre-existing POP3 account too.

I think gmail is still BETA. Maybe that means that they haven't figured out yet whether they will make enough money from the ads to justify offering it to everybody instead of by invitation.

They sure don't make much money from me because I almost never read my Gmail by browser and the emails themselves have absolutely no inserted ads.

DH

Judy G. Russell
July 14th, 2006, 04:00 PM
As hush-hush as Judy is on this, it seems she must have some super-secret conversion program! ;)Heck no! But I do have some excellent text editing programs (TextPad and NoteTab Pro being two of them) that let you do things like delete blocks of info all at once so, for example, you could insert a set of tabs before the pipe character (the | thingy) and then simply delete all of those comments all at once in one block. It's also easy to do some universal search and replace things to take out CServe's internet: addressing and the like. It mostly -- not entirely, but mostly -- a matter of thinking through what the final list needs to look like and then planning how to get there using the excellent features available.

Lindsey
July 14th, 2006, 06:37 PM
If you ever need to do this kind of thing from an Internet cafe (or at the library) when no spreadsheet software is installed locally, you can still do it online at http://spreadsheets.google.com

All you need is a free gmail account.

Oooooh, neat! I have a gmail account, but I didn't know about the online spreadsheet. I'll have to check that out.

--Lindsey

davidh
July 14th, 2006, 07:09 PM
Oooooh, neat! I have a gmail account, but I didn't know about the online spreadsheet. I'll have to check that out.

--Lindsey There are a number of features of google spreadsheets that I haven't tried yet.

But one that looks really interesting is the ability of two or more people to work on the same spreadsheet simultaneously *and* text chat about it in the same window.

DH

Lindsey
July 14th, 2006, 10:56 PM
But one that looks really interesting is the ability of two or more people to work on the same spreadsheet simultaneously *and* text chat about it in the same window.
Hmmmm, and with the IRS plugged into the conversation via the NSA no doubt...

--Lindsey

davidh
July 15th, 2006, 01:08 AM
Hmmmm, and with the IRS plugged into the conversation via the NSA no doubt...

--Lindsey Well, Yahoo, Google, and MSN all caved to the PRC.

OTOH Google is suing the gov't on net neutrality and I don't think Bush's cronies are selling human kidneys yet. So I'm not going to cry in my beer (more likely iced tea) yet.

DH

dgermann
July 15th, 2006, 09:55 AM
Judy--

O Guru! I do not see how inserting a couple of tabs before the pipe allows you to smoke (couldn't resist!) out all the comments.

O Guru, please 'splain it to me! ;)

davidh
July 15th, 2006, 11:01 AM
Judy--

O Guru! I do not see how inserting a couple of tabs before the pipe allows you to smoke (couldn't resist!) out all the comments.

O Guru, please 'splain it to me! ;) First you have to think about what you are starting with, namely the ML file, and then think about what you want it to end up looking like.

Then you also have to think about whether you want to use commas or tabs to delimit fields from one another. Personally I lean toward commas because I can actually see them (hard to tell blanks from tabs).

Generally tab delimiters are not too bad if the original file is ALREADY nicely aligned in columns. If not, then commas may be a better delimiter.

Judy also has some nice text editors that allow you to select text in a plain text file by columns or block (rectangular areas) in addition to just words, lines, and paragraphs. Therefore with her editor, if the pipes were vertically aligned then you could select all the stuff to the right of the pipes and including the pipes themselves and delete them in one click, so to speak. (Even all 500 some lines.)

If you don't have such an editor, you can do almost as easily by changing all the pipes to commas and then importing into a spreadsheet program and deleting ALL the column(s) on the right hand side of the spreadsheet.

Then "save as" from spreadsheet program as CSV (if you are using commas).

Then you can take the resulting CSV file which is text only and edit it with a text editor AGAIN. At this point only names and email addresses should be left.

The problem now is how to get rid of the <> brackets and the double quotes AND end up with a single comma separating the name from the email address.

You will have to think carefully about how to do this with search and replace with your text editor. It's not hard. Just plan ahead so you don't get stuck in the middle and have to start over.

Once you only have a single comma per line sitting exactly between the name and the email address, you can import the CSV into a spreadsheet program again and add the required empty columns. You can even copy and paste columns from one column to another (to change the order of the columns if needed).

Remember that columns A, B, and D must be empty.

C is the name

E is the address

Before doing ANY of this I would highly advise typing up a two entry CSV file totally by hand as you think Evolution should eat it without vomiting

e.g.

, , friend1, , friend1@cs.com
, , friend2, , friend2@cs.com

If you can't get something simple like that to go into Evolution OK then you need to think up a whole nother plan.

FYI , in a spreadsheet, rows are often also called records, and columns are called fields.

You also might rather run a two record test starting with

friend1, friend1@cs.com
friend2, friend2@cs.com

importing it into EXCEL and see if you can use EXCEL or OO to make it into

, , friend1, , friend1@cs.com
, , friend2, , friend2@cs.com

by adding columns at A, B, and D, just to get the hang of it.

Actually this kind of messing around with editors and spreadsheets can be thought of as just another kind of programming. Since if you plan it out carefully, and write your plan down in plain english, then 90% of the work is done as far as making it into a real text manipulation algorithm for that specific conversion requirement.

DH

Judy G. Russell
July 15th, 2006, 06:00 PM
O Guru! I do not see how inserting a couple of tabs before the pipe allows you to smoke (couldn't resist!) out all the comments.O Guru, please 'splain it to me! ;)It's really easy. Once you move all the comments evenly to the right of all the addresses, you select and delete that block. In the attached image, I have moved all the comments to the right by tabbing them over, then selected them. One hit of the delete key and that entire block of text is gone, and nothing else is affected.