This Crafty Robot Can Write in Languages It’s Never Seen Before | WIREDSkip to main contentMenuWIREDSECURITYPOLITICSTHE BIG STORYBUSINESSSCIENCECULTUREREVIEWSMenuWIREDAccountAccountNewslettersSecurityPoliticsThe Big StoryBusinessScienceCultureReviewsChevronMoreExpandThe Big InterviewMagazineEventsWIRED InsiderWIRED ConsultingNewslettersPodcastsVideoLivestreamsMerchSearchSearchMatt SimonScienceMay 27, 2019 9:00 AMThis Crafty Robot Can Write in Languages It’s Never Seen BeforeAfter training to hand-write Japanese characters, the robot could then copy words in Hindi, Greek, and English just by looking at examples.Atsunobu KotaniSave StorySave this storySave StorySave this storyThe AI Database →ApplicationRoboticsTechnologyMachine visionRoboticsAmong the many things we humans like to lord over the rest of the animal kingdom is our complex language. Sure, other creatures talk to one another, but we’ve got all these wildly complicated written languages with syntax and fun words like defenestrate. This we can also lord over robots, who, in addition to lacking emotion and the ability to not fall on their faces, can’t write novels.Atsunobu KotaniAt least not yet. Researchers at Brown University just got a robot to do something as linguistically improbable as it is beautiful: After training to hand-write Japanese characters, the robot then turned around and started to copy words in a slew of other languages it’d never written before, including Hindi, Greek, and English, just by looking at examples of that handwriting. Not only that, it could do English in print and cursive. Oh, and then it copied a drawing of the Mona Lisa on its own for good measure.Like walking on two legs, handwriting is one of those seemingly simple human charms that is in fact elaborate. When you write a word, you have to know where to put down your pen, how long to draw a line and in which direction, then pick up your pen, sometimes mid-letter (like with a capital A), and know where to put it down again.Matt Simon covers cannabis, robots, and climate science for WIRED.So to get a kid to write, you can’t just show them a sample and set them loose—you have to give them instructions on how to form each letter. “They give you these little algorithms for what strokes to do and what order to put them in to make the character,” says Brown University roboticist Stefanie Tellex, who developed the system with Atsunobu Kotani, also at Brown. “And that's what our algorithm is learning to do.”Their learning system is split into two distinct models. A “local” model is in charge of what’s going on with the current stroke of the pen—so aiming in the right direction and determining how to end the stroke. And a “global” model is in charge of moving the robot’s writing utensil to the next stroke of the character.To train the robot, the researchers fed it a corpus of Japanese characters, and provided information about how the component strokes of a character are supposed to work. “From that, it basically learns a model that looks at pixels of the image and predicts where it needs to go to start the next stroke, and then where it needs to move while it’s drawing the stroke to reproduce the image,” Tellex says.LEARN MOREThe WIRED Guide to RobotsThen they decided to try to confuse the hell out of the robot by writing hello on a whiteboard in Hindi and Tamil and Yiddish—all of which use unique scripts. Incredibly, the robot could eyeball each with machine vision and write its own copies of the words, even though it only ever trained to write Japanese. Also, they showed it English cursive in addition to print, and it handled both fine.Then a gaggle of kindergarteners visited Tellex’s lab. Surely, the robot couldn’t recognize and replicate their … suboptimal handwriting? Yeah, it copycatted them with ease. “Just to watch it reproduce the somewhat wobbly writing of these little 6-year-olds, it was just incredible, having never seen that before and never trained on that,” Tellex says.Atsunobu KotaniSurely, the robot wouldn’t be able to copy a rough sketch that Kotani did of the Mona Lisa on the whiteboard? Well, this robot is not so easily confused. “That was back in August, and that picture is still on our whiteboard in our lab,” Tellex says.But nobody’s perfect. Because the researchers trained the robot on modern Japanese, which is written left to right, the system could generalize to English, which is written in the same direction. But it didn’t do so hot with languages written right to left.Still, it’s a remarkable demonstration of the interconnectivity of languages, many different scripts that nevertheless come from the same human (and now robot) hand. And it’s a step toward opening up a new line of communication between humans and machines. Maybe not so much in the near term, but perhaps one day humanoid robots could leave us handwritten notes, as opposed to having them spit printouts from their bodies. Ideally not something ominous like defenestrate! defenestrate! defenestrate!More Great WIRED StoriesInside the hybrid digital-analog lives of childrenThe Chernobyl disaster may have also built a paradiseInside China's massive surveillance operationBluetooth's complexity has become a security riskI'm mad as hell about Square's shady automatic emails🏃🏽‍♀️ Want the best tools to get healthy? Check out our Gear team's picks for the best fitness trackers, running gear (including shoes and socks), and best headphones.📩 Get even more of our inside scoops with our weekly Backchannel newsletterMatt Simon was a senior staff writer covering biology, robotics, and the environment. He’s the author, most recently, of A Poison Like No Other: How Microplastics Corrupted Our Planet and Our Bodies. ... Read MoreContributorXTopicsroboticsRead MoreTop Sony Coupons: 45% Off Sony Headphones, WF-1000XM6 Earbuds, and Sony CamerasUpgrade your setup with Sony’s newest releases. Save on industry-leading noise-canceling audio, and pro-level Alpha cameras.Harry RabinowitzParamount+ Coupon Codes and Deals for August 2026Save on streaming with the latest Paramount+ promo codes and deals, including 50% off subscriptions, free trials, and more.Molly HigginsCanon Promo Codes: 30% Off for September 2026Save an extra 10% or 30% with Canon coupon codes, plus up to $1,600 on cameras, printers, and more this September 2026.Scott GilbertsonTop Shark Promo Codes for September 2026Shark makes some seriously powerful vacuums, from handheld vacs to steam mops. Don’t miss $100 off, 10% off, and more limited-time coupons from WIRED. Nena Farrell Corsair Discount Code: Up to 50% Off for September 2026Upgrade your gaming setup or PC build for less with these verified Corsair coupon codes, student discounts, and refurbished deals.Luke LarsenWalmart Promo Codes: Up to 65% Off for September 2026Score $10 off with our Walmart promo codes and coupon, and shop flash deals up to 65% off today.Molly HigginsTop Chewy Promo Codes: $20 Off September 2026Explore Chewy coupon codes for $30 off, $20 off your first order $49, 50% off pet food, and more September 2026 discounts.Molly HigginsTop AT&T Promo Codes: $50 Off This September 2026Whether you’re looking to upgrade your internet or get the latest phone, we’ve got you covered with our selection of AT&T coupons and deals.Kat MerckTop Alo Promo Codes and Alo Yoga Coupon Codes for September 2026Refresh your workout wardrobe. Save on buttery-soft leggings, chic streetwear, and activewear essentials with verified Alo Yoga discount codes, free shipping, and loyalty program perks.Boutayna Chokrane Top Nomad Goods Promo Codes: Get 25% Off in September 2026Save up to 25% on Nomad Goods accessories such as Nomad phone cases, Nomad wallets, and more in September 2026.Molly HigginsGroupon Promo Codes: 60% Off in 2026Unlock up to 60% off, 40% off packages, and 20% off with top Groupon coupon codes. Explore coupons to save extra on travel getaways, holiday gifts, spa days, and event tickets.Molly HigginsTop VistaPrint Coupons: 30% for September 2026From personalized gifts to business essentials, WIRED can help you save with our selection of VistaPrint promo codes.Molly HigginsWIRED is obsessed with what comes next. Through rigorous investigations and game-changing reporting, we tell stories that don’t just reflect the moment—they help create it. When you look back in 10, 20, even 50 years, WIRED will be the publication that led the story of the present, mapped the people, products, and ideas defining it, and explained how those forces forged the future. WIRED: For Future Reference.More From WIREDSubscribeNewslettersLivestreamsTravelFAQContact UsWIRED StaffWIRED EducationEditorial StandardsArchiveRSSSite MapAccessibility HelpReviews and GuidesReviewsBuying GuidesStreaming GuidesCouponsAdvertiseManage AccountJobsPress CenterCondé Nast StoreUser AgreementPrivacy PolicyYour California Privacy Rights© 2026 Condé Nast. All rights reserved. WIRED may earn a portion of sales from products that are purchased through our site as part of our Affiliate Partnerships with retailers. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Condé Nast. Ad ChoicesSelect international siteUnited StatesLargeChevronItaliaJapónCzech Republic & SlovakiaFacebookXPinterestYouTubeInstagramTiktok

Researchers at Brown University, including Atsunobu Kotani and Stefanie Tellex, developed a system where a robot was trained to replicate handwriting across numerous unseen languages by analyzing visual examples. The foundational training involved teaching the robot to hand-write Japanese characters, upon which it subsequently learned to copy words in various scripts, including Hindi, Greek, and English, simply by observing examples of the handwriting styles. This project highlights the intricate nature of handwriting, which requires understanding not just the appearance of letters but also the underlying algorithms governing stroke formation and sequencing.

The process of teaching the robot necessitated the creation of a learning system divided into two distinct models to manage the complexity of writing. A local model is responsible for governing the current action of the pen, focusing on determining the correct direction for the stroke and how the stroke should terminate. Concurrently, a global model manages the larger movements, dictating where the robot's writing utensil should be positioned for the subsequent strokes within a character. To train this system, the researchers fed it a corpus of Japanese characters along with specific information detailing how the component strokes of a character are structured and function. Through this process, the robot learned a predictive model that assesses pixels in an image to determine where the next stroke needs to begin and where the utensil must move while drawing the stroke to reproduce the observed image.

The experimental results demonstrated the robot's remarkable adaptability. When challenged with writing in unique scripts such as Hindi, Tamil, and Yiddish on a whiteboard, the robot successfully utilized machine vision to eyeball these characters and produce accurate copies, despite never having been explicitly trained on those languages. Furthermore, the system extended its capability to English, handling both print and cursive styles effectively. The robot also demonstrated the ability to reproduce complex visual input, such as copying a drawing of the Mona Lisa, indicating sophisticated spatial and visual reasoning. While the system generalized reasonably well between left-to-right written languages like Japanese and English, the researchers noted limitations when dealing with scripts written in a right-to-left direction. Nevertheless, this endeavor serves as a significant demonstration of the underlying interconnectivity present across diverse languages and scripts originating from the same human hand. This research points toward opening up novel lines of communication between humans and machines, potentially leading to future applications such as humanoid robots leaving handwritten notes.