I guess I work in a strange field, well worked in, in that when you leave your current job it is often assumed neigh expected that you will be contactable and will respond to requests to complete work, collaborate on papers, process structures etc. Other than you get your name as an author on the paper you do this in your own time and at your own expense. Does that seem fair?
It is a tricky situation to handle politically but it is an important part of how academia works. However here comes the rub. Normally within a month of leaving your post your email gets shutdown, closed and deleted. With a normal draft manuscript running to about 30 MB in size not ever email service can cope with sending revisions back and forth. In fact what about those last revisions you had in your last weeks email? What about all the conversations in that email? What to do?
The obvious answer is find a new host for your emails. If you are smart you do this whilst you still have access to your institution/work server and can do a direct transfer from there to your new email provider. I choose google to host all my work email simply because I have after 10 years built up about 3 gigabytes of email in total.
Ok, so you have an email provider now what?
So you got your new email address and provider but you failed to act quick enough to transfer your email directly from A to B. So hopefully your email is all stored locally in point C. This is more common than you think because we all tend to use cached IMAP or pop services for our email (ignore exchange) which leaves a local “offline” copy. That is fine but you want to have access to it more readily on the go or whilst you are replying to “work” correspondence. You need to get that email back into the “cloud”. But how?
I thought at first this is simple I open my email client up and just simply drag and drop from one folder locally to the remote folder. Yes it was easy but unfortunately a little too easy. The transfer resulted in all the date flags being reset and made the emails all appear as new. This included emails from 2003 - NUTS.
Ok so drag and drop won’t work what else can I use?
A quick internet search later and this appears to be a much wider problem than I first thought and thankfully there are at least two solutions for pushing local mail files up into GMail.
First I should state I have two step authentication activated on my account so I had to create an authorised app via the Google “security” centre. I also had to activate IMAP on my GMail account. Again very simple located in the GMail-IMAP/POP settings. I also created a label called** imported**. Well I did for my tests. But quickly had setup a more complicated label structure to cover my various previous work mailboxes for example OLDWORK > PapersInProgress becomes –box OLDWORK/PapersInProgress, etc.
I started out by trying “2gmail.rb” a small ruby script/program written by Pedro Mota but I wasn’t having any luck. It seemed to fail to authenticate correctly and I gave up.
(Updated - 27/06/2013 Pedro - emailed me shortly after my blog post went live to point out why the ruby script did not work for me and quote “Just a reminder, since Mail class was removed from the 1.9.3 standard library, the script must be executed under Ruby 1.8.7.” He updated his blog post to mention the need to use Ruby 1.8.7.) —
Obviously I’ve now completed my mammoth gmail upload but it is always better to know more than one way to achieve an objective so Pedro’s email was very much appreciated._
I am more comfortable with python so I moved on to “IMAP Upload” written by OZAWA Masayuki. This proved to be very successful.
With the simple command:
<span style="line-height: 1.714285714; font-size: 1rem;">python imap_upload.py </span><span style="line-height: 1.714285714; font-size: 1rem;">--gmail --box imported</span><span style="line-height: 1.714285714; font-size: 1rem;"> oldwork.mbox </span><span style="line-height: 1.714285714; font-size: 1rem;">--error oldwork.err</span>
….waiting for a while I watched my Gmail label (or folder if you want) fill and the lines of successfully upload files increase.
I hit a few snags. Mainly files greater than 25 MB in total are not allowed on Gmail apparently but I seemed to only get rejections with files greater than 30 MB. Using the error catcher those emails got stored in an error mbox generated by imap_upload. This is good because if I have a 2 GB mbox I don’t want to keep it because I have 100 MB of trapped emails with attachments greater than 30 MB.
I know this is not a very informative post it is more a thank you to Ozawa-san for his great code. However I should finish off the process. Once all my mail folders were uploaded I deactivated the authorised app and shutdown the IMAP access. I don’t need it for now. I then wiped out the mbox folders keeping the err-files as these contained emails not uploaded.
Job done and so is this blog post.
Apparently Google and a lot of other internet companies, Microsoft, Apple, DropBox, Yahoo, etc, etc have given the NSA access to all their user data (link to Huf post article) I would like to say sorry to the poor NSA analyst who had to read through the 3 GB of emails from the 10 years of email correspondence from three jobs. A very dull read indeed. Erm sorry - if we ever meet I will buy you a beer. I am also sorry that I messed up your tracking of my movements by upload said email which surely will be rather confusing.