Hi. Consider becoming a paid subscriber if you aren’t? This post is free. The mid-month news links are for patrons only, along with the new Esoteric & Weird section. I open with a short article on something weird or cool, and then you get a roundup of AI art tools and creative NLP, with a focus on narrative and games. Plus monthly sf book/tv/game recs! TITAA #49: Secret Missions at LUSHTravel Diary Gen - Real-time & 3D - Funny Diagrams - Narrative - Magic Ink
I’m a day late because of trying to finish my NaNoGenMo (National Novel Generation Month) entry for the year. I used LLMs and Google APIs to create travel diaries of weird missions in European cities, combining pictures and story. The combo of the missions and random site reviews is pretty funny. For example, in Berlin at one restaurant: “Enticed by their business lunch offering - if coordinating my missing-person hunt isn’t serious business, I’m not sure what is.” 😃 It’s like a bad tv ad! Some random juxtapositions were great — like this occult circle trip to Venice:
Technical detail-wise: I used Google Maps/Places/Streetview. The LLMs rewrite randomly selected business reviews as journal entries and social media posts. (Yes, one of them posted “I’m in Venice for a secret occult circle meeting!”) There are morale and money settings, and morale goes up or down by the rating on the place you visit or the random weather. Yes, GPT4 writing style is as mediocre as ever, but it did a great job at combining review details with secret missions. The output examples are here, if you’re interested in looking for fairies in Iceland, missing persons in Berlin, secret occult groups in Venice, or ghosts in Barcelona. I left in the original location review text so you can compare to the journal entries GPT4 generated from them. A couple administrative announcements: The mid-December news link section is paywalled as usual, including the added weird & esoterica bits; as is the end-of the year “Best Books/Games/TV Reviews” roundup post. Otherwise, business as usual! TOC (will be links on the web/app):
AI CreativityThere’s been a flurry of activity in creative AI in the past 2 weeks. Some of the trends have been “real-time” image generation (enabling new tools and workflows), improved video generation or animation, 3D meshes and gaussian splatting improvements, and improved control models for image gen output. There have also been a bunch of super-resolution tool announcements but I’m not covering them here. Let’s go! Real-time Image GenStable Diffusion SDXL “Turbo”: Outputs a 1-step sample image. There’s a slight delay on the clipdrop demo, as can be seen in my video linked below, but it’s pretty cool. It feels like directing a scene? Except there is some real art in prompting these models, as we know, and doing it in a long growing sentence isn’t the best method. But it’s fun to direct it. LCM, which blew everyone away, was using 4-steps. (Code.) Try their clipdrop app demo.  If you want a lighter-weight model SD-Turbo, they also have a 1-step version that works with SD 2.1 tooling here. Meanwhile Adobe (with MIT) also just released a paper on 1-step gen. LCM (Latent Consistency): Drawing tool DEUS code for LCM fast image generation (h/t Dreamingtulpa). Could swap in SDXL Turbo I assume. And here’s a collection of LCM model demos on Huggingface. Video Gen NewsGood grief, so much work in this space! Now we have Stable Diffusion Video too! Announced here, code and models available. Last week’s news, hah. The demo on Replicate produces very…. weird results? For non-photoreal imagery? I mean, not in a bad way, in this case. I started with a still from Orpheus, and went to hell (to be clear, I LOVED THIS) - this is a 3 second clip watered down to an embeddable gif:
The SEINE video generation project has code up now, and I thought the demo was not bad, actually, if given photoreal imagery. When I tried it on the Orpheus image above, it was surreal but less insane than SVD. I am most interested in the SEINE transitions, which that HF demo doesn’t support yet. ⭐️ “Breathing Life into Sketches” by Gal et al — love this! Code coming.
Looking forward to Pika Labs new 1.0 beta release, their video is good. 3D Gen & Pseudo3D
In company tool news, Spline, for 3D web stuff, is getting better and better and will support AI features soon; and also now handles 3d gaussian splats! Blockade Labs is testing their mesh-generation from pano generation now, but I haven’t paid for it to try it yet. It reminds me of the LucidDreamer one above. Tree Asset Generator on gumroad, a plugin for Blender: “Combining geometry nodes, procedural textures and shading, and hand-painted leaves textures, this Stylized Tree Generator allows you to create easily beautiful stylized trees with a high customization, suitable for both Blender beginners and meticulous creators.” (Probably h/t Alex Champandard.) You can even grow your own forest.
🌐 This is clever work from Google (AnyLens) on a model that will produce warp lens effects including equirectangular panoramas and spheres. I guess someone has to reimplement it so we can play with this? Meanwhile here’s a repo for an SD model trained to output panos, SD-T2I-360PanoImage, based on a paper (Diffusion 360), which Camenduru also did a colab for if you want. Tooll3 - via Chris Ried’s newsletter. An open source reactive 3d node-based rendering thingy. Shaders, audio reactivity, etc. Misc and Webby and DiagramsVisual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models by Geng et al. Remember the “controlism” fad a few weeks ago (I covered it here) and the upside-down image generation trick? This is a whole paper on tricks like this including puzzle generation, etc. Absolutely fab. There is code and a colab.
The “Make it More” DALLE-3 meme activity(?) is pretty funny. As someone noted, the “most” is almost always in space or cosmic. Concept Sliders for strength of effect (like “curly hair” or “age” or “cluttered room”). With code and colab, but the sliders need to be pre-trained—which isn’t that hard. I wasn’t totally convinced by the cluttered room — here’s a theoretically more cluttered haunted Victorian room on the left vs. original:
And ZipLora—any subject in any style—from Google folks. Implemented very quickly with their blessing, I gather. This is “a [v] dog in [s] style” enabling. Generative Line Shader Background — I’ve been really enjoying Daniel Velasquez’s mailing tips on webgl arty things. This is a recent post. He has live renders and code on the site and teaches courses.
Mazes - Jo Wood’s Observable notebooks on maze generation and such, for learning/teaching. Excalidraw code. On their live site, they they have a limited text-to-diagram tool. I enjoyed it but got some zingers, like this timeline of the twisty time travel series Dark (you don’t need the details, but this is not twisty):
I asked for a node-link diagram of friends and enemies in Lord of the Rings, and my favorite bit was the “Everyone” at the bottom:
Wireframe-to-Code: Excalidraw also has a wireframe-to-web-code tool like the tldraw one that excited people this month, but you need to plug in your OpenAI api key. You do in all the demos of wireframe to code. You can also use a “Build It” button in Figma again with your OpenAI api key. Also see screenshot-to-code. And Grimoire, a “GPT” for website generation. Interactive Flowcharts (code) from Potsdam folks. “This interface concept of an interactive flowchart proposes the combination of linear audio narration with non-linear open-ended exploration, offering a novel way to understand and navigate complex issues. The interactive flowchart is designed to be inviting and engaging, allowing viewers, readers, and learners to follow a guided tour through a flowchart and pursue their own paths through its contents.”
Games Adjacent & Narrative GenMartin Pichlmair posting on procgen in games in Space Dog Chronicles. A game about finite-state automata challenges (Steam). Some of you will be pulled right into this. Reverie, AI Tools for Unreal 5. Been loving his demo videos on tsfka Twitter. ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games appearing at EMNLP now. Agents: “Learning few-shot imitation as cultural transmission,” in Nature by Bhoopchand et al. Not really about “cultural transmission,” but few shot imitation, about agents communicating. These agents are in GoalCycle3D, a 3D env built in Unity. 🌏 Via Hrbrmstr’s Daily Drop newsletter, Azgaar’s Fantasy Map Generator. There is a nice web demo that allows you to edit a lot of details. I’ve posted other fantasy maps before, but think I missed this one! It feels very Dwarf Fortressy in its procgen, including even “random encounters” with generated characters.
Jon Ingold’s books based on Heaven’s Vault are available now! “Blabrecs: An AI-Based Game of Nonsense Word Creation” by Max Kreminski and Isaac Karth, at NeurIPS. (Reminds me of some of Allison Parrish’s work.) In city generation news… 😅 Welcome to the Bog, a Horrible Place to Live in Cities: Skylines. In the new Aftermath games journal. It is funny, mainly because the residents of this uber-awful slum love it anyway. “The only amenities within the district are a dump and a few coal power plants. I tried adding a monorail that leads to nowhere around the perimeter of The Bog for some noise pollution but apparently it doesn't make any and now they just use it.” Hunter, who made it, says, “I created it as a fun little project to fuck over some simulated people but now I've grown to love the Boggites.” 🕸 There have been a few articles about New York in Spiderman 2. One, “Spider-Man 2’s Brooklyn Is A Beautiful, Empty Condo,” also in Aftermath. The failure and success probably comes down to how and where procgen was used in the workflow pipeline (not at runtime). This is a useful video from AI & Games’ Tommy Thompson on how it was built and here’s the text post about it. (He references a lot of GDC talks.) Narrative Papers/ToolsNarrative: “Improving Pacing in Long-Form Story Planning,” Wang et al. Also at EMNLP now. They train a model to detect concreteness to improve plot generation at discourse level. They use GPT4, and the DOC gen system from Kevin Yang. Code. Sabre Narrative Planner - a loving code release (albeit in Java) from Stephen Ware, for non-commercial use. There is a collection and report on benchmark problems.
An interesting aspect of this tool, relevant to character agents, is that it allows for intentionality, conflict, and theory of mind in its planning. Here’s the original paper. “AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort,” by Wang et al, code soon. Generating story + images with input characters. “Story visualization aims to generate a series of images that match the story described in texts, and it requires the generated images to satisfy high quality, alignment with the text description, and consistency in character identities.” Related, see “Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology,” Halperin and Lukin. “We collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences.” And they analyse them. “On the Automatic Generation and Simplification of Children's Stories,” by Valentini et al. Works on the problem that the readability and lexical level of children’s story generation is not great yet. “GRIM: GRaph-based Interactive narrative visualization for gaMes,” by Leandro et al at Microsoft. Man, I love this. But there is no code I can find. It has it all: generation of stories, linking, graph layouts. Grim “generates a rich narrative graph with branching storylines that match a high-level narrative description and constraints provided by the designer. Game designers can interactively edit the graph by automatically generating new sub-graphs that fit the edits within the original narrative and constraints.” They’re using GPT4 of course, and note that, “On closer inspection, we find aspects of the narratives that still require improvement. The generated storylines could be more varied.” If you haven’t thought about Ancient Rome yet today, here’s Dracula set in Rome with various plots, auto-generated, on their website.
Secret of Monkey Island: Amsterdam (by @levelsio) or how to create your own ChatGPT image+text-based adventure game. (Gist with prompts.) See all the NaNoGenMo projects for 2023! Thanks to Hugo as always for moderating it, he’s still reviewing them even now. I’ll go into more detail on the rest in the next newsletter. NLP & MLLearning/Teaching:
Some cool performance/access developments: gpt-fast (to go with fast-segement-anything) from the Pytorch team, and Llamafile, a one-click-model-and-launch of small fast models (using llama.ccp) for multiple platforms (h/t Simon Willison). The proceedings of the Computational Humanities Research conference 2023, which is next week in Paris and I’ll be attending! DATA: A new updated parquet dump of all of wikipedia and all langs is on HF. It’s 68G. I don’t quite know how to get just English? It should be easy, right? Perplexity is offering API access to their LLMs that are regularly updated with news. E.g., the day I saw this, I asked if Kissinger was still alive and got a good reply. I’ve been having nice little educational chats with Perplexity’s iphone app at night, I like it for giving you useful references. Games RecsI’m playing sf classic Prey and loving it. Thanks to Dan Nguyen for the mention. I am on “story mode” but it’s still combatty, just like Control — and has other similarities, like the administrivia and clues you collect in the office-building-space station about what went bad here. There are also amusing No Man’s Sky-ish crafting mechanisms for ability upgrades that I didn’t expect but enjoy. I love crawling in maintenance tunnels, I will not lie. (I’m only 1/3 in, why am I writing this instead of playing it.) VR: On the holiday sale, I picked up 7th Guest (I’m still Quest 2ing). I never played the original. It’s basically The-Room-style puzzles in a creepy old mansion with narration by creepy people. But I do love the mansion. My favorite detail, worth the price of admission, is shining the magic lantern on the oil paintings and seeing how they “really” look. I love it. Someone had a lot of fun making these: it’s glowing eyes, sometimes hidden Lovecraftian monsters, blood, murders, or ghosts. An example:
TV Recs⭐️ Scavengers Reign on HBO+. Animated series, this was as beautiful as folks said. If you’re into alien biology, both horrific and lovely, strong rec! Game vibes of the planet in I Was a Teenage Exocolonist and visually Sable (I really need to play this). One Night, Australian drama (Guardian review here). I wasn’t sure I’d watch after ep 1, because it’s about the aftermath of a teen rape, but I ended up sucked in by the performances and story. 20 years after the event, one of 3 friends writes a book about her friend’s rape. It’s published anonymously, but people recognize all the events and characters and it drags up buried secrets. (CW obviously: sexual assault.) Loki s2. Not sure what to say here! Despite not buying premises and having many plot questions, I still loved watching Tom Hiddleston and Owen Wilson strut around in a bromance of period tuxes. I never got the Sylvie romance as more than one-sided? Beautiful set design was chewed or at least nibbled. The last 2 episodes solidly pulled me in, though, and I ended a bit traumatized. Ymmv. For All Mankind s4 ongoing. I am still here for this… despite wanting Ed retired sooo badly. The below-decks speakeasy on Mars is a fantastic plot point. Three-Body the Chinese 30 episode version. Yes, I made it through 30 hours. I don’t really recommend, given how padded it was (it took 10 minutes for Ye Wenjie to walk inside the observatory, there was so much snow, music, staring up at it, being emotional etc, and don’t get me started on the looking at the sun through fingers scenes). But the game parts are excellent, and there are some startling film sequences: the destruction of a ship with nano fibers is astounding (and horrific). Book Recs⭐️ Ink Blood Sister Scribe, Emma Törzs (fantasy). Fab read, well-written for adults. Magic books need ink blood, it seems. A family secret keeps Esther on the run every year, leaving her sister at home guarding the special books, resenting her absence. Strangers with their own books (specifically from a British estate’s “Library”) are coming for the collection. There are fantastic magic mirrors in this! (CW: blood.) The Running Grave, Robert Gailbraith (mystery). The 7th Cormoran Strike book (yes, I know who writes them). I needed a travel book, and I couldn’t put it down. Robin goes under cover in a cult. The tv show also continues to be very good. (CW: cultish sexual abuse including minors and more.) The Sun Down Motel, Simone St. James (ghost/mystery). Very good combo of haunting stuff and/or serial killer. A young woman takes a job as the night clerk at a creepy motel where her aunt disappeared years earlier. A Canticle for Leibowitz, Walter Miller (sf classic). I finally read this… it has 3 parts, set in different times since a nuclear war has decimated the world. An abbey that preserves knowledge from before the disaster faces hostile forces. There is a nice bit of folkloric magical realism in part one and bits in the other parts (also humor), but the fundamental question—“will we keep destroying ourselves over and over?”—was depressing. There are virtually no women in this and a lot of insane warlords. Chaos Terminal, Mur Lafferty (sf). This is book 2 of the “Midsolar Murders” featuring the woman cursed by murders happening around her and a strange ability to solve them. She had moved to a sentient space station with interesting aliens to avoid human deaths. The aliens remain interesting, but this was less tight than the first one and some of it was telegraphed. PoemThere is snow, now— A thing of silent creeping— And day is strange half-night . . . And the mountains have gone, softly murmuring something . . . And I remember pale days, Pale as the half-night . . . and as strange and sad. I remember times in this room When but to glance thru an opened window Was to be filled with an ageless crying wonder: The grand slope of the meadows, The green rising of the hills, And then far-away slumbering mountains— Dark, fearful, old— Older than old, rusted, crumbling rock, Those mountains . . . But sometimes came a strange thing And theirs was the youth of a cloudlet flying, Sunwise, flashing . . . —D’Arcy McNickle, from “The Mountains”, h/t Poem-A-Day If you made it through to the end again — Hope your holiday/winter season is going okay so far. It’s chilly and rainy in France, big surprise! Time for cookies and TV and heated throws. Best, Lynn (@arnicas on the sfka twitter, mastodon, and bluesky, where I have some invite codes if you get in touch) You’re a free subscriber to Things I Think Are Awesome. If you’re a fan, and you want to support me in writing this, consider becoming a paying subscriber in order to get the complete mid-month updates including the new esoterica section, or buy me a coffee to express your appreciation.
|
пятница, 1 декабря 2023 г.
TITAA #49: Secret Missions at LUSH
Подписаться на:
Комментарии к сообщению (Atom)
Комментариев нет:
Отправить комментарий
Примечание. Отправлять комментарии могут только участники этого блога.