The art of algorithms: How automation is affecting creativity

ANALYSIS:

“Drawing on your phone or computer can be slow and difficult — so we created AutoDraw, a new web-based tool that pairs machine learning with drawings created by talented artists to help you draw,” wrote Google Creative Lab’s “creative technologist,” Dan Motzenbecker, earlier this week.

AutoDraw is one of Google’s artificial intelligence (AI) experiments, working across platforms to let anyone, irrespective of their artistic flair, create something super quick with little more than a scribble. It guesses what you’re trying to draw, then lets you pick from a list of previously created pictures. “So you can’t draw? No worries!” is the general idea here.

First up, AutoDraw is a super fun tool that gets increasingly addictive — that much is clear. But what’s also clear is that the tool is more a display of AI smarts than it is a tool to improve your artwork, because it would be just as easy to embody the exact same functionality within a text-based search engine. I mean, why bother drawing a crap dolphin with your finger when you could just type in the word “dolphin”? Because it wouldn’t be nearly as much fun, and Google wouldn’t get to show off its fancy new toys.

A few days after Google debuted AutoDraw, it revealed some other research its scientists have been carrying out, designed to enable computers to generate simple sketches using artificial intelligence (AI). In effect, they trained a recurrent neural network (RNN) on sketches that real people made, which emanated from an experimental app called Quick, Draw! that launched last year (again … it is really fun). The app tells you to draw things, like a giraffe or a butterfly, and then it guesses what you’ve drawn. So what Google is doing is training machines to sketch like real people, with all the line overlaps and crappy squiggles included.

What this helps demonstrate is the growing crossover between art and algorithms. But does this hint at a future where humans have little incentive to be creative at all?

The rise of the fourth industrial revolution

As part of the so-called fourth industrial revolution, millions of jobs will be lost to automation, according to a recent World Economic Forum report. The net loss is expected to be as many as five million jobs by 2020, though of course a whole bunch of new jobs will be created, including positions in IT and data science. Jobs such as manufacturing and production are expected to be heavily affected, while another recent report indicated that more than 100,000 legal jobs will be automated over the next 20 years.

But art… art is sacred. Art is an expression of human sentiment and emotion. Computers stand zero chance of consigning human creativity to the history books. Right? Well, maybe. We’re already seeing the early signs that art will be disrupted by machine intelligence and automation.

Why bother learning to paint a landscape or pay someone to sketch your newborn when you can download Prisma to your smartphone and transform your snapshots into ultra-realistic pieces of art in seconds? Prisma, for the uninitiated, uses neural networks to analyze each photo and then applies a style the user selects. And it really is rather good.

“Based on deep-learning techniques, we redraw the image from scratch,” said Alexey Moiseenkov, Prisma Labs cofounder, in an interview with VentureBeat last year. “We analyze tons of photos and get the typical forms and lines, then take a style and draw your picture with those lines in a taken style.”

The point here isn’t that these tools are better than human creators. The point is that such tools are pretty good just now, and they’ll only get better. If someone can press a couple of buttons to get an instant “hand-drawn” family portrait, using little more than a DSLR camera, tripod, and a Prisma-style AI image-rendering app, why would they bother employing the services of a professional artist?

It’s not beyond possibility that artists and art retailers will one day have to sell their services based on their authenticity — “100% hand-painted pictures” could become the only visible marking that separates human creations from those produced by machines.

But technology’s algorithmic arm stretches far beyond that of photography and art and into other creative realms.

In design

For years, automated web design services such as Wix and Weebly have offered novices an easy-to-use web development platform that makes it simple to build HTML5 sites using drag-and-drop tools rather than code. For basic websites without much deep functionality, such tools work fairly well. But the formulaic, simplistic, template-based approach leaves much to be desired, which is why professional designers and developers still manage to eke out a living.

Last June, Wix launched an automated web design service built on artificial intelligence, called Wix ADI. Using data garnered from its existing user base to feed into this new AI offering, the “creator” basically answers a few questions and provides the platform with cues as to what theme the website should be based on and what category it exists in, and then Wix pulls in relevant photos, words, and layouts based on the business type and location.

“Wix ADI isn’t just a new website builder — it sets a new market standard for web design,” said Wix ADI head Nitzan Achsaf at the launch. “We have been at the forefront of this market for nearly a decade, and now as one of the leading AI technology providers, we will make website creation accessible and easy for everyone.”

Wix promises that no two websites will look the same.

Other similar AI-focused web design platforms have blossomed in recent times and raised significant venture capital funding, including TheGrid, which has been operating its AI smarts for a few years already, and B12, which launched a similar proposition in beta last year with more than $ 12 million in funding.

The credibility of DIY web- and app-design tools that promise to turn “noobs” into designers and coders has been questioned for years. And now that AI is going the extra mile to remove any further effort from the process, it will only ruffle the naysayers’ feathers even more. But the usefulness of such tools really depends on what the purpose of the website is. Why pay for a professional designer and developer when you can hit a few buttons and have a simple, informative, Google-friendly site made with next to no spadework?

Again, the point here isn’t that the machines are now good enough to replace professionals in building fully functional websites and online services. The point is that AI is encroaching further into creative professions and, more importantly, it’s improving all the time.

In music

Could an algorithm ever be able to produce something as exquisite as Lennon & McCartney, Jagger & Richards, or even Mozart? Maybe. But probably not, at least for a while.

Back in September, headlines across the web screamed that the first AI-written pop song had been made. It made for alluring headlines, but it wasn’t strictly true. Sony researchers, using specialist Flow Machines software, were able to train a system on different music styles using a gargantuan database of songs. Then combining “style transfer, optimization and interaction techniques,” the system is able to compose music in any style.

So what we have here is a song called “Daddy’s Car,” written in the style of The Beatles. And hey, it’s not too bad.

However, a more accurate description of this composition would be that it was “AI-assisted.” French composer Benoit Carré wrote the lyrics (which are pretty nonsensical) and arranged the song — all the computer did was identify commonalities across this style of pop music and provided Carré with the parts to play around with. Sony’s researchers have actually been working on AI-assisted music creations for a few years already, and an entire album of such music is expected later this year.

Sony isn’t the only company dabbling in this field. Last year, Google announced Magenta, a project from the Google Brain team that’s setting out to discover whether machine learning can “create compelling art and music.” And earlier this year, the internet giant released a working interactive version of AI Duet, an app that lets you play a virtual piano with accompaniment from a computer system that riffs off what you play.

Elsewhere, London-based startup Jukedeck is working on an AI-powered music composer that writes original music completely on its own volition. Aimed at video creators on the hunt for original background music, Jukedeck has been training deep neural networks to understand how to compose and adapt music, with the end-user able to customize the sound they’re looking for.

All the guitar bands, DJs, and orchestras of the world can perhaps rest easy for now. While computers will improve at “songwriting,” artists’ biggest worry for the time being is how to make money in the age of on-demand streaming. Speaking of which….

Spotify snapped up music intelligence and data platform Echo Nest back in 2014, and off the back of that acquisition has been doubling down on its music recommendation efforts. The star of the show is Discover Weekly, a personalized playlist of music built around songs you’ve previously listened to on the platform.

In effect, Spotify analyzes your history and meshes it with the listening behavior of others to see what songs commonly appear next to each other, then based on this information it recommends new music. And it is more than pretty good — it is pretty excellent. While Apple is banking on human curators via the likes of Apple Radio, Spotify is arguably winning the music-recommendation battle using algorithms and automation.

What’s most interesting about this is that it is infinitely more scalable than a human DJ’s ability to recommend new music. Playlists built on algorithms are always tailored to the individual, while human recommendations will always have biased subjectivity weighted against it that will never appeal to everyone at all times.

Similarly, Shazam analyzes song structure to tell you what the name of the song is and who performs it. All you need to do is hold your phone up, tap a button, and voila. It really is a great way to discover new music and build up a library of tunes that you encounter on your day-to-day business, be it in a shop, at a football stadium, or while watching TV. Such technologies make everyone an expert, without having to become an expert. You don’t need to know anything except how to tap a button to identify a song, while Shazam links in directly with Spotify and iTunes to make it easy to stream or buy music.

Together, the likes of Spotify and Shazam could put a sizable dent into the knowledge-powered smarts of music writers and DJs around the world. People have instant access to all the information they need on the music they hear around them. And why listen to the top 10 charts on the radio, or read the top 5 albums of the week in the NME, when you know that Spotify has all the best new music? And why turn to your music-obsessed buddy to ask what the name of the song in that TV advertisement is when you can just Shazam it?

With algorithms at work, the need for human knowledge and expertise diminishes.

In writing

It’s difficult to envisage a time when a machine will be capable of crafting a best-selling novel, but lord knows geeks have been trying to make that happen for a while. It’s not overly difficult to create something that is formed of words and roughly comprehensible in parts, but generating something with a proper narrative that flows beautifully from start to finish and is infused with wit and passion — well, that could be a long way off yet.

But we are already at a stage where machines are producing journalistic content (for want of a better phrase). Last summer, the Associated Press (AP) revealed it was expanding its baseball coverage with automated stories generated by algorithms through a partnership with Automated Insights. The AP had worked with Automated Insights for years already, generating thousands of computer-generated corporate earnings reports.

Automated Insights uses artificial intelligence to analyze big data and transform it into stories. Chicago-based Narrative Science offers something similar, with a specific focus on business intelligence for the enterprise, or “data storytelling,” as it puts it.

Here’s an AP report from a baseball game in the New York-Penn league, powered by Automated Insights.

STATE COLLEGE, Pa. (AP) — Dylan Tice was hit by a pitch with the bases loaded with one out in the 11th inning, giving the State College Spikes a 9-8 victory over the Brooklyn Cyclones on Wednesday.

Danny Hudzina scored the game-winning run after he reached base on a sacrifice hit, advanced to second on a sacrifice bunt and then went to third on an out.

Gene Cone scored on a double play in the first inning to give the Cyclones a 1-0 lead. The Spikes came back to take a 5-1 lead in the first inning when they put up five runs, including a two-run home run by Tice.

Brooklyn regained the lead 8-7 after it scored four runs in the seventh inning on a grand slam by Brandon Brosher.

State College tied the game 8-8 in the seventh when Ryan McCarvel hit an RBI single, driving in Tommy Edman.

Reliever Bob Wheatley (1-0) picked up the win after he struck out two and walked one while allowing one hit over two scoreless innings. Alejandro Castro (1-1) allowed one run and got one out in the New York-Penn League game.

Vincent Jackson doubled twice and singled, driving in two runs in the win.
State College took advantage of some erratic Brooklyn pitching, drawing a season-high nine walks in its victory.

Despite the loss, six players for Brooklyn picked up at least a pair of hits. Brosher homered and singled twice, driving home four runs and scoring a couple. The Cyclones also recorded a season-high 14 base hits.

This story was generated by Automated Insights (http://automatedinsights.com) using data from and in cooperation with MLB Advanced Media and Minor League Baseball, http://www.milb.com.

And here’s an earnings report in Forbes, powered by Narrative Science.

Over the past three months, the consensus estimate has sagged from $ 1.25. For the fiscal year, analysts are expecting earnings of $ 5.75 per share. A year after being $ 1.37 billion, analysts expect revenue to fall 1% year-over-year to $ 1.35 billion for the quarter. For the year, revenue is expected to come in at $ 5.93 billion.

A year-over-year drop in revenue in the fourth quarter broke a three-quarter streak of revenue increases.

The company has been profitable for the last eight quarters, and for the last four, profit has risen year-over-year by an average of 16%. The biggest boost for the company came in the third quarter, when profit jumped by 32%.

Earnings estimates provided by Zacks.

Narrative Science, through its proprietary artificial intelligence platform, transforms data into stories and insights.

Such reports won’t be winning any Pulitzer prizes yet, but they’re perfectly readable and the algorithms are constantly improving. There’s no evidence that machines will be capable of producing something akin to Dickens or Proust, but who knows what another 10 years’ worth of data could do to improve their writing smarts?

“A machine will win a Pulitzer one day,” noted Narrative Science’ chief scientist Kris Hammond, in the Guardian. “We can tell the stories hidden in data.”

While fears abound that algorithms will kill off human journalists, figuratively speaking, the AP has previously stated that embracing machine-written stories is more about expanding its coverage than replacing journalists. Through this method, it can cover many more Minor League Baseball games it would not have previously covered, simply by using data provided by news and statistics body Major League Baseball Advanced Media (MLBAM).

“Augmented content was never intended to replace human-generated content,” explained Joe Procopio, Automated Insights’ chief innovation office, in an interview with VentureBeat. “It’s another tool, another arrow in the journalist’s quiver, so to speak, and it should be used in places where it can take a lot of the data science and number crunching off the journalist’s plate. That frees up the journalist’s time to be able to do more of the investigative and reasoning work inherent in their jobs.”

What will ultimately decide whether an artistic endeavor is replaced by an algorithm or set of algorithms, in a business setting at least, is whether it’s more efficient. The question is: Does it save time and money without compromising on quality?

“There are basically two boxes that need to be checked when deciding to use automation to tell a story,” added Procopio. “One, is the data available to write something compelling, and two, is the business case there — in other words, does automation save enough time and resources to make it worthwhile?”

So can a machine be trained to amend its style of writing depending on whether it’s writing an earnings reports, a baseball review, or an obituary? Absolutely — this is already happening. Could a machine write a review of a music gig? Or write up an interview? Potentially, but it all comes down to the quality of the data the platform is given, and whether it’s actually cost effective to train a system to become efficient at such write-ups.

“Automation can be used when writing the types of pieces you describe — feature, interviews, reviews, etc., where automation makes sense,” continued Procopio. “How much of the piece should be automated depends on the scope of the piece.”

What’s emerging here is that such tools could be more about assisting the journalist than replacing them. It might not make sense to attempt entire computer-generated write-ups of a music gig, for example, if it already requires a human to attend the gig and form an opinion. But it may make sense to use a machine to fill in the gaps in the final review, or even to format it properly. For example, automation could generate paragraphs on a particular band’s sales and downloads, or maybe ticket sales, through tapping existing databases that contain up-to-date information. It’s not really important whether a human or a machine finds and compiles such data, so long as it’s accurate, but using an automated approach could save a journalist a lot of time.

Found in translation

Away from the journalistic sphere, the global translation and interpretation industry is reported to be worth around $ 40 billion. And contrary to what some may think, the process of converting words and meanings between languages requires a great deal of creativity. Often words or sentiment don’t convert well between languages and vernaculars, leaving the translator to trawl the nuanced depths of their linguistic abilities to communicate the intended meaning in another tongue.

Historically, machine translation tools have had a bad rap, but they are getting better. It’s now possible to plug any foreign-language newspaper article into Google Translate and receive a pretty faithful interpretation in another language, though there are many colloquialisms that will still trip up the best machine translation tools out there. Google has started using its AI-based neural machine translation across more of its public-facing services.

Skype also has a real-time voice translation tool, which lets you speak with someone (verbally) in a foreign tongue such as Japanese, in real time. Skype Translator uses AI smarts such as deep learning to train artificial neural networks, meaning it should improve over time as it listens to more conversations.

Any business worth its salt would not rely 100 percent on machine translations for mission-critical communications with customers. But we are certainly fast approaching a stage where machines can be called upon for less important stuff, and perhaps used in tandem with a proofreader to correct mistakes and clarify any ambiguities made by the machine for use in more important communications.

So, as with Automated Insights, we could have a situation where 100 percent automation is used in some instances where it makes sense, but in cases where the nuanced understanding of a human is needed, the two would work in conjunction with each other.

Where we’re at

It’s clear that the threat from automation to human jobs is real for many industries, and that includes the creative realm: streaming services that serve you the perfect playlist, apps that turn a family photo into something straight from Van Gogh’s easel, real-time translations and interpretations, robot-written news reports, and websites created automatically simply by answering a few questions.

This leads us to one stark question. Creativity is a core defining human trait, something that truly separates us from the machines, so where is the incentive to get creative when all these tools out there are setting out to save us from doing it ourselves?

There are a number of positives here. If a computer was to get as good as, or better than, humans at drawing in a natural style, then it could become the teacher, or assist an artist in their own creative process. Plus, there is a strong line of argument that says that people will always have a creative streak and will want to do things themselves. If you can click a button to turn a photo into a work of art, where is the fun in that?

And that is something that humans will never lose: a desire to have fun and make things themselves. Whether they will be able to get a job off the back of it in 20 years time is another question, of course.

When technology is constantly “fixing” human errors, be it a typo in a Word document or a wonky line in a drawing, humans may gradually lose the ability to perform certain creative tasks without computer intervention. It’s no longer necessary to remember facts, or phone numbers, or routes to your grandma’s house in the next town, because we know it’s all instantly accessible through a phone. This surely has an impact on a brain’s ability to remember things. Similarly, if kids grow up with tools to “help them draw” on their phone or computer because it’s “slow and difficult” otherwise, this can’t bode well if it becomes the norm.

But let’s not get too carried away. Machines have yet to prove they’re up to the job of many creative tasks; all they’ve shown so far is they can chip away at the edges — and even then they still need human assistance. Highly creative projects such as writing novels, writing investigative journalism, or penning an entire album of original music with heartfelt, meaningful lyrics — it’s difficult to see a time in the near future where computers will trump humans.

A good example is this cool little short sci-fi film produced last year, called Sunspring. It stars real actors, but the script was written by a machine. It was inspired by Alphabet’s AlphaGo AI system beating a pro player at the age-old strategy game Go.

The script for the short film was authored by a “recurrent neural network called long short-term memory, or LSTM for short,” according to a report in Ars last year. It is actually really funny, and makes little sense, but it serves as a reminder as to how far behind machines are in terms of creating genuine works of art that humans would wish to enjoy at scale.

It’s also important to distinguish between artificial intelligence and “algorithmic intelligence.” The former is more about computers being able to think, understand, and adapt in way a human might, while the latter is more about using mathematics to help people and machines work together.

Phil Tee is chairman and CEO of Moogsoft, a company that specializes in bringing algorithmic intelligence to enterprises — Moogsoft basically helps them adopt algorithms to address mundane operational tasks. He told VentureBeat:

Artificial intelligence is the ability for computer systems to perform tasks that traditionally have required human intelligence, such as visual perception, speech recognition, decision-making and language translation. Algorithmic technologies such as Algorithmic IT Operations (AIOps), on the other hand, leverage mathematics to help operators navigate dynamic, and highly unpredictable settings such as enterprise IT environments. There isn’t anything artificial about algorithms.

And this is a key point. Using algorithms to predict what music you’ll like on Spotify or what movies you should watch next on Netflix is smart for sure, but it’s not creative in itself. It may be better at doing its job than a human is, but it doesn’t exist as part of “the arts.” So while we’ll see businesses increasingly turn to algorithmic intelligence to optimize and streamline their operations and differentiate themselves from the competition, art itself may not be directly under threat.

But will we ever reach a stage where a computer could write a completely coherent book, song, or movie of its own volition?

“Absolutely, but the advances necessary are quite imposing,” added Tee. “The typical neural network today has roughly hundreds to tens of thousands of neurons, which makes it even less intelligent than a sea slug, which has 18,000 neurons in its brain. This journey to a creative thinking machine is vital, but a long one. Perhaps we should be more focused on intelligence as an aid to creativity rather than a replacement. After all, creativity probably is ultimately what defines humanity.”

Art needs humans, and humans need art. Machines may increasingly help the two work together, and it may even replace some jobs, but as one of our defining characteristics, humans and art will continue to be inseparable.

READ MORE: How to draw better