Archive for the 'Technical details' Category

Jul 10 2010

Migrating From Zimbra to Google: How We’re Doing It (Part 1 – Notifications)

A little while back I wrote up a short evaluation of the Google Migration Tool – highlighting the pros and cons I found. After a ton of iterations, I think we may have stumbled upon success. (Knowing my luck the process will now fail miserably because of this post).

The migration process is comprised of 3 main components:

  • End-user notifications
  • Folder renaming
  • Migrating mail

The notification and folder renaming processes are pretty straightforward. The actual migrating of mail is rather complex and I’ll cover the details to that in part 3.

Since Brandeis has chosen to take a rolling migration approach, we have the luxury of moving only a select number of users at a time. When we made the move to Bmail, we found that moving a single department at a time worked well. At least with this method, we can catch a set of users who work in close proximity to each other and can look to one another for some first tier support.

The secret to moving departments on a one-by-one basis is to first create a master spreadsheet containing all users who have a Bmail account, the department, Brandeis affiliation, the number of messages in each account, the number of folders as well as number of Google illegal folders, and the size of the mailbox (we have no quota so this # is important). Sure creating this spreadsheet only captures numbers at a moment in time but it at least gets us a rough idea of how large a user’s mailbox is or how many e-mails they have.

Next we choose a department and work with them on selecting a migration date. Not knowing any one department’s work schedule, this coordination has proven invaluable. It also helps to catch the one off cases where a user may be traveling and having to change to a new e-mail system while on the road would cause some confusion.

Once the department or group has agreed upon a date, a CSV list of their Brandeis UNet IDs is uploaded to our Google Migration Application. Doing so makes the following things happen:

  1. Once uploaded, key dates are calculated.
    Dates from the Google MIgration Application

    Dates from the Google MIgration Application

    1. Add Date – the date they are added to the application.
    2. Reminder Date – 2 business days before they are scheduled to move
    3. Migrate Date – the actual day the migration will take place
    4. Close Date – the date their Bmail account will close
  2. A kick-off notification is sent at 4 PM on the Add Date. This e-mail contains general information about the move, links to documentation, as well as a pre-migration checklist of things to do.
  3. On the evening of the Add Date, a separate process is run which checks a user’s folders for length, illegal characters, Google specific system labels, and folders with 2 or more adjacent spaces. If any are found, a notification is sent to the user highlighting each of these folders and why they will not migrate to Google. (More on this process will be covered in Part 2).
  4. On the Reminder Date, a reminder e-mail (surprise!) is sent to the users at 9:30 AM. This is more a less the same e-mail as the kick-off notification, with some slightly different verbiage since the migration is only 2 business days away.
  5. At 11 PM on the Reminder Date, the folder check process is run again, basically repeating the same e-mail as before. However, if the user fixed all folders from the initial notification, a ‘Thanks for renaming your folders!’ e-mail is sent instead.
  6. At 5 PM on the Migrate Date, the folder check process is run last time, only this time it renames any folders which are still illegal. An e-mail with each folder, before and after being renamed, is sent to the user.
  7. At 5:30 PM on the Migrate Date, the migration begins. We do some magic to calculate the number of e-mails a user has and provide them a rough estimate of how long the migration will take. Last I checked we assume a rate of 2,000 e-mails per hour – so if you have 4,000 e-mails the notification will state 2 – 3 hours to complete.
  8. During the migration, the user may be notified of e-mails which couldn’t be migrated to Google. If this is the case, they will be sent an e-mail with a link to download all e-mails in a folder which could not be migrated, along with the date, subject, from address, and message ID of each individual e-mail which did not get migrated.
  9. Once the migration is complete, a final notification is sent with the next steps – most importantly how to point your mail clients or mobile device to Google instead of Bmail.
  10. No notifications are sent on the Close Date, rather, the Bmail account is silently closed via an automated process.

All notifications and their subject

All notifications are stored in our Google Migration App, making it easy to see who got notified and when. The app also allows users to be contacted directly from the interface, again storing the notification in the database.

Notifications are all made possible by:

  • Turbogears, the Python based web framework which the Google Migration Application was written in
  • Cron to automate and schedule when the notifications run
  • MySQL which stores all the users, dates, notifications, etc. from the Google Migration Application
  • Python + MySQLdb and smtplib (and other misc. modules) which all the scripts are written in

In Part 2 I will explore in a little more detail the folder renaming process, which like everything else, has undergone a few iterations before we got it ‘right’. Finally, in Part 3, when I have a few *free* hours, I’ll document the actual mail migration process.

5 responses so far

May 12 2010

Google App Status Page

You may have noticed the side bar feed from Google’s app status page, but the feed doesn’t do the status page any justice.

The status page provides the current status for all of Google’s services and up to date information on any issues they are having. This is a good place to check first if you are having problems accessing any of the Google services.

Comments Off on Google App Status Page

May 11 2010

People’s name in Google

The Google account sync gets information about people (such as their name and email address) from the Brandeis directory and stores it in Google. Unfortunately there seem to be some issues with names. Primarily what’s noticeable is that when someone has a space in their first name, all that’s ending up in Google are the characters before the space!

Doing a quick search, there are literally thousands of people at Brandeis with a space in their first name! Sounds like something we should rectify, eh?

I’m working with my co-workers on improving how names gets from Brandeis to Google. The good news is it’s getting us thinking and working towards improving some identity management items. While it shouldn’t be a huge deal to correct, I didn’t anticipate this problem and the time/effort to correct it!

If your name looks incorrect in Google please be patient as we work to setup a better way of sending Google our names.

UPDATE: We are now syncing names in a better manner! If your name still looks incorrect let us know.

Comments Off on People’s name in Google

May 05 2010

Multiple Google Notifiers

One of my favorite features of Bmail was the Zimbra Toaster. It was a simple, but helpful pop-up notification whenever I received a new e-mail. Super useful when not browsing my e-mail client, but annoying (and potentially embarrassing) when giving a presentation.

Google’s toaster equivalent is the Google Notifier (also available for Windows I believe). I’ve been using this nifty little menu bar app for my personal account for as long as I can remember but was concerned I couldn’t run a second instance for my Brandeis Google Apps account. After a quick Google search, I stumbled upon this article from From the site:

  1. Duplicate the Google Notifier application.
  2. Select the duplicate, control-click on its icon, and choose Show Package Contents from the pop-up menu.
  3. Navigate into Contents, and then open Info.plist in an editor.
  4. There is a property in that file called CFBundleIdentifier, with the value Change the property’s value instead.
  5. Save your change and quit the editor.

Apparently you can also change the icon, but you need developer tools to do so. I think I can live with the same icon for now.

Comments Off on Multiple Google Notifiers

May 01 2010

Maximum Label Length: 40!?

Published by under Gmail,Technical details

Are you a user with tons of folders, with folders within those folders, and folders within those, and so on? Or maybe you prefer to keep a flat folder structure, but use-hyphens-in-your-folders-creating-really-long-folder-names?

User beware!

Google Mail has a 40 character label limit. Plus, users with sub-folders have to eat a character for each ‘/’ in the name. Users have voiced their concerns with the limit, but I think a better approach would be to suggest it.

One response so far

Apr 30 2010

Google Migration Tool. Nice, but …

Published by under Gmail,Technical details

It seems like yesterday I was writing a python program and web based account verification system to migrate our user’s mail from a legacy, UW-IMAP system to Zimbra NE. Is it deja vu?

I am now tasked with migrating mail from Zimbra to Google. How hard could it be? I’ve done it once, how different could this be?

For starters, Google offers a slew of methods for getting our mail over there. The question is, which?
Continue Reading »

One response so far

Apr 27 2010

Lose your Android, iPhone, or Blackberry? No problem — we can wipe it!

Google Apps for Education allows us to use Google Sync – which emulates a Microsoft Exchange sync server. Devices like the Android, iPhone, iPad, and iPhone can use this to connect to Google and sync their mail, contacts and calendars. It also allows us to remotely wipe the device if it’s lost or stolen!

We will have to figure out how we will support this from an enterprise perspective, but it’s a very good improvement over what we have today!

4 responses so far

Apr 26 2010

How to set up Thunderbird for Gmail

Published by under Gmail,Technical details

Nick Ragusa found this post on how to set up Thunderbird to work well with Gmail.

This is a must read for anyone who wants to use Thunderbird!

One response so far

Apr 26 2010

iPhone iPad iPod touch limitations on Google

Are you frustrated trying to get your calendar events to sync with your iPhone? Can’t accept that invitation to your event on your iPad?

Well, there are some things that don’t work in the iPhone OS.

Comments Off on iPhone iPad iPod touch limitations on Google

Apr 25 2010

Delegating Access to your email in Gmail – Updated: Not working

Published by under Gmail,Technical details

Delegating or sharing an email account has always been a tricky option with most mail servers. At Brandeis, we have the courier system, and we allow users to share passwords (not a good idea). With Zimbra, you could share your mail, but there was a caveat that you had to be on the same mailstore.

I thought with Gmail we might not be able to offer this anymore, but wait! Yes we can!!

This is untested and who knows what email clients it will work with.



It appears this is only possible with a Google apps premier account (which we kind of have.) I’ll add this to the list of questions to ask our TAM (technical account manager)

Also see:

2 responses so far

Next »

Protected by Akismet
Blog with WordPress

Welcome Guest | Login (Brandeis Members Only)