#!/usr/bin/env python
# -*- coding: utf-8 -*-
blog = u"""The eye 

My name is Prof. Eric Warrant from the University of Lund in Sweden (blonde hair, blue glasses), and I am here together with Prof. Dan Nilsson (also from Lund) to study the gigantic eyes of the colossal squid. These are truly amazing eyes - in the collapsed state we see here, they measure 25 cm across, but in the living animal they are probably larger, up to around 30 cm in diameter (the size of a soccer ball). These are without doubt the largest eyes that have ever been studied (and probably among the largest eyes that have existed during the history of the animal kingdom), and it is tremendously exciting for Dan and I to be here in Wellington.

The eyes of cephalopods (squid and octopus) are very much like our own, "camera eyes" that contain a single lens that focuses images onto a retina lining the concave rear surface of the eye. We have removed the lens from one eye, and as in all cephalopods, it consists of two halves. In the pictures showing two "lenses" we are in fact looking at the two halves of a single lens. They are somewhat degraded, and these two halves probably represent the higher refractive index core of a larger lens - the jelly-like coating around the lens (of lower refractive index) has most likely disappeared. When this squid was alive, the lens was almost certainly spherical and possibly of a size similar to an orange (ca. 80 - 90 mm diameter).

The "optic lobe" of each eye is the part of the brain that processes the visual information coming from the eye. Even though we have not yet had the chance to look at the optic lobe in this Colossal squid, we did have access to the optic lobe of a considerably smaller Giant squid (Architeuthis dux). Its optic lobe was the size of a small sausage - larger in size than the entire visual cortex of a human (the visual cortex is our "optic lobe"). The optic lobe was also much larger than the remainder of the Giant squid's brain, and shows just how important vision is to these huge squid.

I suppose the question on everyone's mind is why these squid have such enormous eyes? Firstly, by having huge eyes it is possible to have huge pupils, and a huge pupil (in our Colossal squid probably around 80 - 90 mm across) allows the eye to collect every last photon of light in the incredibly deep and dark waters where it lives (ca. 1000 m below the sea surface - human visual threshold would occur at a depth of around 500-600 m). Large eyes also allow the possibility of high spatial resolution (the ability to distinguish spatial detail), although it is possible that neural mechanisms in the optic lobe sum signals from groups of neighbouring photoreceptors, thus making the visual "pixels" larger, but also much brighter. Thus with enormous eyes, and a large and complex optic lobe, giant deep-sea squids, like the colossal squid we are studying here, have the potential for advanced vision in the profoundly dark depths where they live. This ability would be potentially useful for many tasks, including the detection of prey and predators, and for seeing each other.
""".lower()
ap = u"""Study: Colossal squid has biggest eyes in the world
WELLINGTON, New Zealand (AP) — Marine scientists studying the carcass of a rare colossal squid said Wednesday they had measured its eye at 27 centimeters (10.8 inches) across — larger than a dinner plate and the biggest animal eye on earth.

One of the squid's two eyes, with a lens as big as an orange, was found intact as the scientists examined the creature while it was slowly defrosted at New Zealand's national museum, Te Papa Tongarewa. It has been preserved there since being caught in the Ross Sea off Antarctica's northern coast last year.

"This is the only intact eye (of a colossal squid) that's ever been found. It's spectacular," said Auckland University of Technology squid specialist Kat Bolstad, one of a team of international scientists brought in to examine the creature.

"It's the largest known eye in the animal kingdom," Bolstad told The Associated Press.

The squid is the biggest specimen ever caught of the rare and mysterious deep-water species Mesonychoteuthis hamiltoni, or colossal squid. It is 26-feet long and weighs almost 1,089-pound, but scientists believe the species may grow up to 46 feet long.

"I think they will be entirely visual predators with eyes like that, and the lens was truly phenomenal," said another scientists, Steve O'Shea.

"This is the largest eye ever recorded in history and studied," said Swedish professor Eric Warrant of the University of Lund, who specializes in vision in invertebrates. "It has a huge lens the size of an orange and captures an awful lot of light in the dark depths in which it hunts."

They can descend to 6,500 feet and are known to be aggressive hunters.

While scientists have yet to confirm the squid's gender, Bolstad said it was "probably a female, females grow larger than the male, we believe."

It is thought to have enormous strength, with swiveling hooks mounted at its tentacle ends to snare prey. Further up the tentacles, fixed hooks with three razor points help it hold onto large prey as they are sliced into thumb-sized pieces for swallowing.

The colossal squid's chief predator is believed to be the sperm whale.

Another squid specialist, Japanese scientist Tsunemi Kubodera of Japan's National Museum of Nature and Science, said he had tasted a piece of colossal squid from a smaller specimen. He said it was edible but the flesh contained ammonia and tasted bitter.

The plan is eventually to put the squid on public display in a 1,800-gallon tank of formaldehyde.

Bolstad said the creature would be preserved in formalin solution later Wednesday — a process that would take two weeks.

Te Papa museum communications manager Jane Keig said its website had 100,000 hits Tuesday as the carcass study was broadcast on the Internet, and 1,800 people on average were watching the event at any time.
Copyright 2008 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.
""".lower()

def depunctuate(text):
	from string import punctuation
	import re
	punctuationRE = re.compile('['+punctuation+']')
	return punctuationRE.sub(' ', text)

blog, ap = map(depunctuate, [blog, ap])

blog_words, ap_words = blog.split(), ap.split()

def avg(sequence): return sum(sequence)/float(len(sequence))

def lengths(words): return [len(word) for word in words]

print "Average Word Lengths"

print "%.2f Blog" % avg(lengths(blog_words))

print "%.2f AP" % avg(lengths(ap_words))

print "\nLongest words"

print "Blog: ",
print ' '.join(sorted(blog_words, key=len, reverse=True)[:20])
print
print "AP: " + ' '.join(sorted(ap_words, key=len, reverse=True)[:20])