Backplane App Now Available!
After months of teamwork from our incredible mobile engineers, the Backplane is happy to announce that we have our mobile app available for download in the iTunes App Store and Google Play. If you thought you were addicted to Paradise City now, just wait until you experience Axl on our app.
Our Backplane app features include:
- Access to all of your communities with a simple swipe
- Stay in touch with Lady Gaga’s most recent selfie or pictures from the Guns N’ Roses’s latest tour stop
- Create and share posts
- Take and upload pictures while you’re on the go
- Send and receive direct messages from your friends in the community
- Search popular topics in the community, like Artwork or Personal Updates
- …and more!
Existing community members should immediately notice the speed of browsing, the feed is faster, switching between information happens at a much faster rate than the mobile web. Members who are in multiple backplane communities will appreciate the simplicity of switching between communities. Our mobile team built the application with our community members in mind the entire time in order to make the Backplane experience as unique as possible without having to abandon any of the features that make our product so special.
We want to hear what you think about our app, so feel free to share your questions, comments, and fan-girl moments with us at email@example.com.
The Backplane is pleased to announce the launch of its latest community, Mandela Project. In partnership with Nelson Mandela’s grandson, Ndaba Mandela, the Backplane is honored to create a community around the revolutionary figure. PandoDaily described Mandela Project as “a social hub built around the idea of giving back to the community. It will also help extened and amplify celebrations around Mandela Day, which itself is meant to inspire change and spur people to take action for the good of their communities.” If you are interested in becoming a part of the movement, please sign up for an invitation at www.mandelaproject.com
Artistry Meets Technology at the #ArtistsHACK
The Backplane’s #ArtistsHACK brought burgers, bands, California weather, and a slew of APIs to Austin on Sunday for our SXSW take over. Hackers came out in droves to build apps dedicated to the future of music, art, video, and collaborative thought on the web and mobile. And once again, we were reminded of just how much we love tech.
The Raptor House was gracious enough to once again let us take over their space at the fabulous St. Cecilia hotel and the adjoining Arlyn Studios in the heart of Austin.
And take over we did. APIs were everywhere and came from:
- MusixMatch, and
- Music Dealers.
Having this many APIs in play meant two things: we saw a ton of variety in the hacks and the demos segment awesome.
Because of the diversity in the entires, our judges had some tough decisions to make. Luckily, our panel of judges was up to the task — the list included Cory Booker (Newark city mayor), Emmanuel Seuge (Head of Global Sports & Entertainment Marketing at The Coca-Cola Company), Judy Estrin (CEO of JLabs), Jonathan LeBlanc (Developer Evangelist at PayPal), Chad Martin (Director Social + Emerging Media at VML), and Richard Jones (Manager of the Pixies and Dead Can Dance).
A Brief Blackout
Because there is never a dull moment during our hackathons, our production crew faced a brief power outage that delayed some of the demos.
Backplane CEO Matt Michelsen took that opportunity to conduct an impromptu Q&A with Cory Booker. Kudos go to them both for keeping the
mood light (even without the lights on).
Overall, there were 20 hacks that came out of the #ArtistsHACK. There was a service that would build you a playlist based on your heartbeat, a Chrome extension that would stream your browser activity to the public, a true hack that allowed you to control Spotify through Siri.
The true beauty of the event, though, was seeing so many hackers come together from across the planet to make products that were not only technologically sound, but artistically elegant as well.
The winning project was a Spotify Concert Browser, which enabled SXSW music enthusiasts to see what concerts were taking place nearby and the ability to listen to top tracks by the artists. Because we like rewarding hard work and determination, winners were lavished with prizes ranging from
- Apple iPads,
- 2 free years of Spotify Premium,
- 100 free .co names, and
- $50,000 in free PayPal transactions.
Hackathon bystanders were also greeted with delicious grub and live music from some of SXSW’s finest — Paper Hotel, New York City Queens, and Chris Hawkes provided live entertainment throughout the day’s events and DJ Reflex capped off the evening. We provided the hacking, you guys provided the entertainment.
Again, we want to thank our sponsors again for making our crazy hackathon a reality: Coca-Cola, NASDAQ, EA, Gunnar Optiks, Raptor House, SideCar, GoPro, go.co, Qualcomm Labs, PayPal, and Music Dealers. Because of you, we are granted the ability to host hackathons and share our passion for innovation in technology.
Until next year, Austin. We’re looking forward to it.
Get Your Hack On
We at the Backplane like to have fun. In between monitoring community interaction and writing scripts in vim, we like to dream. We like to collaborate with geeks just as passionate as we are and try to change the world.
And because (a.) Austin is rad (and weird) and (b.) the people at there know how to build some pretty kick-ass products, we also like hosting hackathons.
We made our SXSW debut last year when we partnered with Spotify at our Managers Hack. The Managers Hack brought hackers from across the U.S. to the St. Cecilia Hotel in Austin in hopes of building an app, a platform, or a piece of technology designed to advance the future of digital music distribution.
In only five and a half hours, the hackers built products that varied from an app that uses Nike FuelBand to build radio stations based on your physical environment to an app that culls through your Twitter feed to build Spotify playlists from the music you’ve shared.
Our judges—who included Troy Carter (Lady Gaga’s business manager) and Jay Brown (president of Jay-Z’s Roc Nation)—chose RequestLine (a democratized party playlist maker) as the first-place winner. We left inspired by the sheer genius that came out of the products that were built.
We knew our hackathon days were not over.
This year at SXSW, we want to pay homage to the artists—the hackers who look for ways to incorporate artistry into their design; the thinkers who see a mediocre idea and grok how to evolve it into an incredible product; and the makers who want to see their creations change how even geeks look at technology.
For you, we present the Artist’s Hack.
In partnership with Spotify, we are asking hackers to build a product that is dedicated to bridging the future of music, art, and collaborative thought. Because we always choose to go big (instead of going home), our hackathon will offer a live webcast with play-by-play commentary from top YouTubers…all while being broadcast to the jumbotron in Times Square.
Cool, huh? We think so, too.
This hackathon would have just been another dream of ours if it had not been without the help of our amazing sponsors: NASDAQ, EA, Gunnar Optiks, Raptor House, SideCar, GoPro, go.co, and Music Dealers.“So, if hacking works like painting and writing, is it as cool? After all, you only get one life. You might as well spend it working on something great.”
— Paul Graham
We like to work on great ideas with great developers interested in creating incredible works of art. Come develop a masterpiece with Backplane and Spotify on March 10th at the Raptor House.
— The Backplane Team
Building an Analytics Portal from Scratch, Part 2
This is the second of a multi-part series written by our lead data scientist, Eli Finkelshteyn, on how and why we built the Backplane analytics portal from scratch. Be sure to check out part 1 as well.
Thanks to everyone who sent me positive feedback about my first blog post. I really appreciated all of it, and it made writing this next part a much better experience for me. — Eli
Now that you know why we wanted to move the bulk of our analytics and logging off Google Analytics and Mixpanel and onto our own platform, here is what we did instead.
Whatever solution we chose, it would need to jive with the following list of priorities:
- All logging data must be saved and be in a scalable format for new data and schema necessities.
- All data must be warehoused somewhere reliable and easily accessible.
- There must be flexible processing and analytics storage layers. We’re going to want to process and slice our data into new charts and visualizations in real time (or in batches, depending on requirements).
- The solution must be reasonably fast and horizontally scalable—we need to be able to handle any future expected load without dying. Success for us means that if the number of active users on our site jumps by more than 100x, the site won’t crash and we don’t have to spend all of our time putting out fires.
- It must be built in about a month of my time. When I was building this, I was the one data scientist Backplane had, and this was just one of my tasks. Start-up mode means no one can go off and spend months perfecting pet projects for fun. Done is more important than perfect.
The system I built accomplished all of these things, and none of the boxes it’s on have so much as hiccupped in the 5 months since. That’s the best endorsement I can give.
The Logging Layer
For the logging layer, the data needed to be
- …stored in JSON (as opposed to something like comma-delimited where the schema is external to the data). What I was logging would probably go through several changes, and the logs needed to be backwards compatible with regards to processing.
- …periodically saved to s3 so I wouldn’t need to worry about losing it, or accessing it. Luckily, we could do that for pretty cheap.
- …able to be parallelized, so that I can tackle fires by throwing in more boxes and have everything decentralized. One box dying shouldn’t break the system.
I researched Scribe, Flume OG and NG, and Kafka.
- Scribe: I worried about Scribe because it’s not a central part of Facebook’s business, isn’t extensively documented, and didn’t seem like it had a very active user community. It’s also no longer an active project at Facebook, which is hugely scary in case we found bugs or needed updates.
- Flume was in a middle state between transitioning from OG to NG which did not sound like they were compatible. NG wasn’t quite finished or being used by many people, and building something on OG knowing full well it would soon not be supported made no sense.
- Kafka had a small user community and could very well go the way of scribe and become unsupported by LinkedIn at any time. The documentation also seemed poor. As a sidebar, this guy who did decide to go with Kafka has made some good points on the topic.
On top of all this, I was spending a huge amount of time hacking and googling just to try to get something basic working with any of these solutions. Forcing these solutions to store data in JSON or to save to s3 sounded like a big pain, especially since the user communities were, again, not very responsive.
That’s when I ran into FluentD. It’s a new competitor in this market, but it allows me to store everything in JSON by default. And with its plethora of plugins, I can have it do writes from most common languages, write to s3, and any number of other tasks. And because it is written in very clean Ruby code, I can easily modify it myself if I ever need to. On top of all of that, the authors actively answer questions in their user community.
With all that going for it, I had a prototype set up and writing the data I needed in JSON to s3 in mere hours. I had been skeptical of using a product that was so new and untested, but the fact that it was capable of doing everything I wanted and do it so quickly — it was refreshing after spending days of frustrating research trying to bend the other solutions to our needs.
So, when I set up a handful of aggregator boxes and load-tested the hell out of it by sending traffic at about 500x what we were currently receiving. FluentD withstood it all, no problem. I was convinced.
Here’s a sketch of our logging layer with FluentD:
The data is written from each of our app servers to a couple of aggregator boxes that then aggregate the data into a few larger files. Those files are then uploaded to s3 on an hourly basis for safe keeping. Simultaneous to that, the aggregator boxes also stream the data they receive to a processing layer for real-time processing.
One of the concerns brought up in the last article was about logging security. We haven’t found a silver bullet here, but we have been able to make things a lot more secure and reliable than what we had with Google and MixPanel. We do this by bucketing logs depending on how much we trust them.
Log Bucket 1: Backend
The most trusted bucket holds the logs generated from our backend. For example, when a user goes in and “likes” a post, a log is generated on our backend only when that action actually happened. Someone could create a bot that goes around liking a bunch of posts to spam our site, but our logs would still be accurate—those “likes” actually did happen (plus, protecting against bots is a completely separate problem).
Compare this to using only Google or Mixpanel, where someone could just spoof the front end calls for “likes” to make it look like a post got liked a lot when in reality it didn’t. That’s a much bigger data validity problem because if that happens even once, it means we can’t trust any of our data anymore since we don’t know where or when such a breach might have happened before.
Log Bucket 2: Logged-in Users
The second most trusted bucket is front-end logs generated from logged-in users. These are bucketed separately from our backend logs in case problems do occur, so they don’t contaminate the backend logs.
The idea for these logs is on every front-end log call we get, we check to make sure that where they’re being sent from has a valid logged-in session, and we know the user ID and the IP address. We throw out outliers on both of these fronts, so if an abnormally high amount of activity is coming from any single IP, or any single logged in user, it’ll be automatically thrown out in processing. That means if someone wants to give us bad data here, they have to create a bot that uses a large number of spoofed IPs, creates a bunch of spoofed user accounts, and pays for cracking CAPTCHAs on all those.
There are also a number of other checks we do on these, but those are more for security through obfuscation. So, even after going to all of the trouble mentioned above, a potential hacker still wouldn’t know if their hack was successful since they never see our data.
Log Bucket 3: Everyone else
The final bucket is front-end logging for users who aren’t logged in. This is a really small part of our logging, and we try not to use it whenever possible. Still, when it’s absolutely necessary, it is completed by doing all of the checks above except for the username checking. Data going here is also bucketed separately so it doesn’t contaminate any of the other logs.
I should note that even our least secure layer of logging (which we try not to use) is still better than logging through Google Analytics and Mixpanel since we throw out the outliers and have full access to all of our own logs. Thus, if something suspicious happens, we can easily and immediately check it out.
The next part in this series on our analytics portal will cover Processing and Visualization. Stay Tuned!
Little Monsters Chat, Scaling, and Future Features…
Access to the Little Monsters community has been unrestricted to everyone on the social web for the past few weeks. Since opening our doors to all Little Monsters the Backplane platform has scaled rapidly, and our development and engineering teams are working round-the-clock to keep up with interest in features such as Little Monsters Chat.
The Backplane team is thrilled at the level of community engagement with Chat and, at present, Little Monsters are logging hundreds of millions of characters each day. Providing a Chat experience that translates 57 languages in real time, empowers everyone in the Little Monsters community to share, connect and communicate about all of the issues and ideals that add value to the global narrative.
As with any new web platform that scales quickly, Little Monsters Chat has experienced the need for incremental development along the way. We are working quickly to evolve the platform and address the considerations that the Little Monsters have brought to our attention. The team has heard and registered all of the community feedback, and we are making changes to improve on the user experience.
The evolution of Little Monsters Chat will include enhanced features and functionality including:
- Viewer mode — creating a view into the stream for all interested Little Monsters to see what people are talking about in any Chat room
- Geolocation — empowering any Little Monster to find friends in Chat rooms who are nearby or in a place of mutual geographic interest
- Mobile — delivering the full Little Monsters experience, including Chat, on any mobile device is a development priority for the Backplane team
We deeply appreciate your input and patience as we scale the Little Monsters community. As we work together to remove boundaries to a new and global communication platform, we look forward to enhancing and improving Little Monsters Chat along with you.
— Team Backplane
Truly Effective Social Design Requires Context
A whiteboard sketch, c. April 2011, by Joey Primiani, envisioning a new type of community platform
Backplane unites people around affinities, interests and movements. Backplane is a new type of social platform aimed at helping niche communities build, host and grow robust and engaging online systems. Currently, the social web is centered around individuals and their friends, we believe that its future surrounds communities and bringing people together who share common interests, passions or industries.
Being a Gaga fan and understanding how important the thriving Little Monster communities, we wanted to keep a very focused design that would showcase user generated content effectively. The waterfall UI is a very visual way to consume a lot of content quickly. We are really inspired by Dieter Rams “10 Principles for Good Design”. Specifically good design is as little design as possible and good design is in the details.
- To have people connected to each other
- Make them feel like the superstar
- Connect on a more personal, intimate, and creative level
- Creative Community of Fashion Designers, Photographers, Artists, etc.
- Social networks are for people who you went to high school with, Backplane is for the people you wanted to go to high school with.
- Be the exclusive source of new content
- Showcase User Generated Content & Media
- Create the closest two-way creative feedback loop platform
- Visualize everything that is a movement, impact, and influence
- Authenticity and legitimacy to the brand
Two things are true with the future of social - it is much more visual to promote self expression and much more real-time to create a great conversation channel for communities. Want to learn more? Email us firstname.lastname@example.org