ChatGPT, LLMs and the unfolding generative AI revolution

So, it seems like the future is here and rapidly unfolding! Ironically I've been struggling to even mention anything about recent machine learning events, here on this blog. Despite AI being such a big feature of its theme on exponential tech changes...

New Scientist style chatbot illustration generated using StableDiffusion

5 days to 1M users! The fastest growing web
  service, ever. From Twitter via YouTube.

To recap: from the middle of 2022, there was simmering online interest in the spectacle of AI generated art via text prompts: DALL-E vs Stable Diffusion.

Also, Google's firing of a senior software engineer who was claiming sentience and personhood for their LaMDA chatbot (e.g. Guardian). Which kicked off nervous discussion.

Then, at the end of the year, OpenAI unleashed ChatGPT-3 to the public, making a huge splash in the process! Machine learning has sustained main stream awareness through 2023, so far, with news hype hitting fever pitch in tech circles, over the last week of announcements. E.g. on YouTube LTT wan show from here and Matt Wolfe.

As talked about in many sources, machine learning has been advancing up a much steeper exponential curve than Moore's law, both GPU hardware advancing faster and ever bigger datasets being trained on. 

Of course, the main reason I've struggled to keep fully abreast of these exciting developments: I'm still struggling with the ME/CFS. As I've lamented on Twitter, this blocked my desired career path into academic AI/robotics. Being forced to give up on 3rd year studies just over 15 short years ago. And in my small amounts of productive time, during the last year, I've been working on informational resources (e.g. Steam guides) and other things in the Core Keeper community. Which is sometimes within my cognitively impaired abstraction limit.

Anyway, I'm aiming for this post to be ongoing, like the previous Covid posts. Updated with my thoughts and new events as they unfold. Because I don't have the executive function to write a big piece with an overarching narrative. If I've ever really achieved that here before, heh.

I've no hope of capturing any of the soap opera of corporate and big-player drama, or even most of the notable offerings. Let alone the technical details of how the systems opera. So really, this is largely an exercise (as always) in showing that I was here and I maybe understood some fraction of what was going on. Maybe an aide memoire for me, like taking notes in class. 

I'm also explicitly aware that the ever-growing LLM training data with likely include this text at some point, too! Which is very meta and a little thrilling to see happening already.

I'll start by catching up on some backdated bits and pieces...

James Lovelock (of 'Gaia' fame) going strong at 97, but is he right to call it quits on saving the planet...?

Screenshot of referenced Guardian article.

In response to this article in the Guardian online:

Yeah, when any headline attempts to be scary by talking of catastrophic climate change consequences by 2100, I personally tune out. All bets are off anything past mid century (see "The Singularity is Near" [2005])... Unless one believes advances in computing/AI and other information technologies are all about to come to an abrupt halt. Well within this century the game will have changed so drastically that additional atmospheric carbon will just be a handy resource for nanotechnological mass fabrication, or an irrelevance.

So I personally identify with much of Lovelocks irreverence, here, regarding the meaning of climate change. Particularly the fantastically flawed ideological foundation of the Greens, which may be badly in need of a rude wakeup call: expecting anything remotely like a return to nature and balance to be plausible?! Anything less than full steam ahead exponential technological progress would be like dabbing the motorbike brakes at the point of daredevil jump takeoff. Steady state 'sustainability' is civilisational suicide. (But as a political party they've otherwise had the best policies I've seen on offer in recent times.)

I don't think that building new nuclear is viable stop-gap anymore, though, we've as good as missed that juncture already. Perhaps a big push in 2004, back when he wrote this linked article: Nuclear power is the only green solution. But by mid 2020s, when the UK's Hinkley point B comes online, it's probably going to seem even more overpriced and ill fitting. (Against the background of plummeting clean energy generation and storage installation costs.)

Coving "100 sq miles of the Sahara in solar panels" as a monolithic solution to all EU's electricity needs is obviously just a mental figure for imagination's sake, so presumably it's just off-handed remark to talk of it being too "easy for terrorists to go and bugger it up". Obvious there's a several fold efficiency benefit to that latitude and local whether. But smaller scale, more localised production will, I expect, win out, in small part because of the security/stability benefits. (Given PV price efficiency continues to improve to the point of making locational efficiency secondary.)

Spiraling out: animated gif, from here.

"A timeline of Earth's average temperature"
XKCD comic 1732.

What does worry me most about climate change is the "20 years before it hits the fan" part (which would be 12 years now, as of 2016), with regards to the truly mass migrations this will force and how politically catastrophic that's likely to be. That is, given how divisive the relatively small number (few million) from Syria, and the regional cluster-fuck, have been for Europe thus far (playing into the Brexit vote). How much war is there going to be?

A definitive end to the 'long peace' (since WWII) is a scary prospect. Even before considering all the (tactical ) nuclear weapons lying about. The horrendous suffering for countless humans (quite possibly even those of us currently living in complacency) is an aside that Lovelock's enthusiasm for the big picture fails to register. So too here, pretty much - ultimately, what matters most is only how much conflagrations might derail technological progress (or wipe us out entirely).

Last year, Venkatesh Rao wrote (in The Atlantic) of how a sufficiently coordinated, effective response to climate change would require a literal war footing, with the kind of personal sacrifice of living standards and freedoms that has previously involved. As for it being "so much cheaper to air-condition the cities [than the whole planet]" (Lovelock Guardian interview 2012) - sure, I love that kind of enthusiasm for minimal effort adaptable ingenuity as the kind of inevitable compromise outcome. ) But where will we be growing the vast majority of our crops for the next few decades? That long tail-off before true factory (vertical) farming production scales up enough? (Or more radical solutions kick in - like Kurzweil's wholesale replacement of human digestive organs.)

Finally, talking about "robots taking over" only in the context of Earth makes very little sense. We squishy, slow meatbags are already aiming at interplanetary colonization, starting in a decade (well, you know, that one guy has certainly been saying so this week). So the meager terrestrial resources that 'robots' who think "one million times faster" that us might command down the bottom of this gravity (an empire of dirt) are likely to be a pretty niche concern by comparison to what already floating around out there...

On: "Knights of Sidonia" Anime (and Related Current Affairs)

"Knights of Sidonia" totes plenty of whizz-bang space battles with giant mecha suits piloted by teenagers repeatedly fighting incomprehensible space monstrosities that attempt to tentacle humanity to death.... But! Push past the facade of these overused anime tropes and be rewarded by a rich, engaging space opera.



Summary - Components of This Well Assembled Pastiche:

There's a strong 'hard sci-fi' undercurrent carrying the story along and underpinning a decent amount of attention to detail.

Then there's teenage awkwardness, of course, and tensions in a kind of high school-ish setting (of the pilot academy). The socially inept protagonist, chosen-one (Nagate), who alone can pull the joysticks of the legendary armour suit to it's full potential... But thankfully it's never as paralytically morose as NGE (Neon Genesis Evangelion).

It is respectably gritty, but not like the gratuitous gore of Akira. It has decent depictions of the brutal suddenness of war. This is brought home via the perspective of sympathetic characters, mostly very young, like the unlucky soldiers of real wars. But, of course, the fantastical space setting and perversely monstrous threats make it psychologically gentler (safer) for an audience to engage with emotionally.

Yuhata Midorikawa's different faces.

It also avoids being too bleak via juxtaposing the relatively low key social tribulations of the pilots, burgeoning on adulthood. Prime example of this is Yuhata. She swings between command responsibilities, gambling everyone's lives, to then be the star-struck, boy obsessed, teen girl. (Also gluing together geeky toy models kits in her spare time.)

There's a definite 'harem' sub-genre element running throughout, adding lighter notes, with support character developement and romantic intrigue. With three main contenders for Nagate's attention, there are few other females encountered who don't also make a play at some point, brood over him or have a twinkle of possible interest. These are mostly relatively subtle, making sense in context, since our protagonist is deliberately set up to be an inspirational hero who necessarily appeared mysteriously from no-where. An attractive combination, I suppose. But then this blank slate POV character with an open selection box of different flavoured lovers meshes rather neatly with the psyche of the core young-ish male demographic. Who would not love to imagine themselves as an overlooked darkhorse hero themselves? (If only they had their moment to shine or step out of themselves!)

Residential interior of Sidonia - caricature of the 
dense, hilly urbanisation of 'spaceship Japan'?

The flip side of all these female characters is that there are a whole lot of complex, strong female characters. In fact, they run the ship, from captain to XO, to commander, chief engineer, dorm mother, genius doctor/scientist. So, I suppose that even if this is basically down to the Manga artist/author enjoying drawing the female form more, then so be it. The result is good overall, although the bias towards showing only female communal 'photosynthesis' room scenes feels a little exploitative.

Also thrown into the pot are the unavoidably ubiquitous anime tropes of instant karma for (accidental) male lechery, always receiving a bloody nose, usually quite directly off a fiery female. Plus an alpha dog bully antagonist with long, grey/white hair and a massive family inheritance. But even he has an interesting story arc.

Finally, the art style is seamless 3D CGI. It's more noticeable in the space action scenes, when things look a little bit Tron - monochrome with neon highlights. But then it blends into a more rustic feel while inside sidonia's city spaces, with familiar cartoon stylings. The characters faces generally appear unnoticeably traditional. There's a retro aesthetic flavour, too, with the uniform outfits: almost storm trooper-esq, with stuffed up gear. There's some disconcertingly high flung, pointy boobage going on, but we'll put down to genetic engineering and/or reduced gravity, and move swiftly on...

The marriage of high tech space mecha, moving city(ies) of giant rusty pipes and cthulhu-ish space monsters is similar to, though far less gordy than, 2013's "Gargantia on the Verdurous Planet".


Original Manga vs Anime:

The anime appears to follow the original manga (comic) publications very closely. Almost shot for shot, with only some minor sequence juggling and tweaks here and there. So while I'm talking about the Knights of Sidonia (KoS) anime here, most comments apply to both.

Given that the manga has concluded already, it's a little tempting now, after watching season 2, to just read the final third (volumes 44-78, or thereabouts), rather than wait while the animation studio is apparently finishing up with a different project. It would save totally forgetting the plot details a second time.

But the video format is more compelling and I often struggle more to figure out what's supposed to be happening in the strips. It's a far more mentally tiring and different experience. At only 20 minutes per episode and a dozen per season, it's probably almost as quick for me to watch as read and digest the corresponding 20 odd manga volumes with 30 or so pages a piece. Plus you get to hear the cute Japanese voices and groovy theme tunes.

There's No Hyperloop in HS2. [£80Bn update]

Elon Musk played the media perfectly, with coy comments about at high speed rail alternative to the proposed LA to San Francisco project, building the intrigue with hints before finally releasing a comprehensive looking 57 page PDF with design and root schematics, costings and explanations. He certainly succeeded in grabbing my attention, and I think everyone should take his proposals very serious, however amazing they may seem; i.e. "Don't be afraid to dream a little bigger, darling."

He makes his PayPal co-founder, Peter Theil (via Confinity), look like a bit of a looser (especially after those losses at Clarium Capital the other year). Musk stuck to creating companies engineering new solutions:

Musk: robot, alien or superhero?

• SpaceX in 2002, the first private company to supply the ISS, delivering a $1.6Bn spacecraft project on time and budget. 
• Tesla in 2003, which makes all electric, domestic use cars that accelerate faster than Ferraris, now expanding rapidly into commodity price ranges.
• And SolarCity in 2006 (technically founded by his cousin from Musk's idea) which is "the largest provider of solar power systems in the United States", with Musk as the largest shareholder he stands to be an *extremely* rich person from this investment alone, given how solar PV is set continue growing exponentially in efficiency and scale over the next couple decades.

I mean, who IS this guy?! A visitor from some technologically advanced, alien civilisation? Or our future? Or JUST genius of a Tony Stark level competence?

So when musk says he can send millions of people per year, safely down partially evacuated tubes at 700mph, solar PV powered, electric turbine trains, riding only cushions of air, for far less than the expected budget of the unimaginatively planned rolling route... you'd best believe him!!