Update with Select using Joins

Using a technique called Path Enumeration, we can arrange hierarchical menus, typically seen on shopping websites. Here's a query to quickly generate the path tree for menu items. Here, it is shown to build paths for children to parent nodes 4 and 6.

UPDATE ADJLIST A INNER JOIN
(
SELECT A1.id, CONCAT(A2.path,A1.id,'/') AS NEWPATH FROM ADJLIST A1
INNER JOIN (SELECT id, path FROM ADJLIST WHERE id IN (4,6)) A2
ON A1.parent_id = A2.id
) A3
ON A.id = A3.id
SET A.path=A3.newpath

See: 

http://stackoverflow.com/a/25328460/1330710

http://stackoverflow.com/questions/1262786/mysql-update-query-based-on-select-query

http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#function_concat

http://stackoverflow.com/questions/14815668/add-to-existing-value-in-mysql-column-using-concat-function

http://www.slideshare.net/billkarwin/models-for-hierarchical-data

maker fair revolution

https://events.webmaker.org/
http://www.whitehouse.gov/the-press-office/2014/06/18/fact-sheet-president-obama-host-first-ever-white-house-maker-faire

"America has always been a nation of tinkerers, inventors, and entrepreneurs. In recent years, a growing number of Americans have gained access to technologies such as 3D printers, laser cutters, easy-to-use design software, and desktop machine tools, with even more being created by the day. These tools are enabling more Americans to design and build almost anything.

New technologies for rapid prototyping – from laser cutters to CNC routers to 3D printers – have dramatically lowered the cost of developing a prototype and starting a business in manufacturing. The ability to rapidly and affordably test, tinker, monitor and customize places a premium on locating production close to American markets and opens new doors to entrepreneurship and innovation in manufacturing. The power of these emerging technologies creates the opportunity for Makers to launch new businesses, create jobs and build the industries of the future."


Ignorance isn't bliss.

Sharding (refresher)

Scaling is an eventuality with any growing business. It comes in many forms. For the online business, database scaling is the first big technical hurdle.

The new generation of webapps demand a near-persistent connection with the database, doing multiple read/writes over a single session; the more LIVE data you throw, the longer "they" stick with you. Couple that with the "viral" aspect, and you know that soon as your app gets noticed, the database demands are gonna surge. Databases are the foremost when it comes to contingency planning.

DBs are moving to the cloud, much like our individual data. Clouds, for all the good they offer, aren't strong when it comes to sustained I/O ops.
A 0.9GB database takes 1 min to load.
A 30GB database takes 10 days to load...

Another aspect is that the newer-gen NoSQL DBs are single threaded. Which means the CPU is only handling a single op at a time. In case of a lock, it could take a long time to resolve. Everyone (communicating with the DB) takes the brunt.

Horizontal partitioning is a design principle whereby rows of a database table are held separately, rather than splitting by columns (as for normalization). Each partition forms part of a shard, which may in turn be located on a separate database server or physical location. The advantage is the number of rows in each table is reduced (this reduces index size, thus improves search performance). If the sharding is based on some real-world aspect of the data (e.g. European customers vs. American customers) then it may be possible to infer the appropriate shard membership easily and automatically, and query only the relevant shard.

Another place sharding can be used is to reduce contention on data entities. It is especially important when building scalable systems to watch out for those piece of data that are written often because they are always the bottleneck. A good solution is to shard off that specific entity and write to multile copies, then read the total. An example of this "sharded counter wrt GAE: http://code.google.com/appengine/articles/sharding_counters.html

If you have queries to a DBMS for which the locality is quite restricted (say, a user only fires selects with a 'where username = $my_username') it makes sense to put all the usernames starting with A-N on one server and all from M-Z on the other. By this you get near linear scaling for some queries.

Resources
[1] http://en.wikipedia.org/wiki/Shard_(database_architecture)
[2] http://stackoverflow.com/questions/992988/what-is-sharding-and-why-is-it-important
[3] http://www.slideshare.net/rightscale/rightscale-webinar-scaling-your-database-in-the-cloud
[4] https://vimeo.com/32541189

Fight Spam, but how?

Been inundated with spammy referrers in the last few months. I'm sure I'm not the only one. I know what it is - backlink spam [1, 2], which means those dirty ad networks generating fake clicks to blogs, to be displayed in the backlink/trackback list, which in turn adds to their PageRank on Google. So wily are these spammers that instead of using their direct URL they are now masking it through t.co and then further wrapping them into StumbleUpon or Facebook or [insert any respectable sounding name] redirects. The blog owners will both be confused and curious - "Gee, who could be linking to me from http://www.filmhill.com/redirect.php?url=http%3A%2F%2Ft.co%2FUEmjqui3"

Sadly, even though I have turned my backlinks off through Blogger settings, these guys won't stop. The TLDs are too diverse to block out. But one common pattern observed among these links are that all use t.co to masquerade. Probably could insert a JS snippet to block any incomings of this nature. But I don't see how that solves anything - Google Analytics will still register the referral since the page is going to be accessed and its going to show up anyways.

97 things

I'm reading a book called "97 things a programmer should know". After the first few pages, it seemed like common sense, so I put it down. It didn't solve my crisis of the stalemate I find myself in. Not only that, it was generally uninteresting.

What I do want to read, however, would be "97 things a programmer wouldn't know", which could introduce some new paradigms to the art of software programming that an average programmer won't be familiar to - after years of being in the industry, I have seen people decaying and falling out-of-date; or "97 things a programmer should forget", which could be a treatise against the prevalent bad practices that every new guy falls into.

Next I pit hopes on "Beautiful Architecture", which I understand as an effort towards creating a "programmauteur", my ideal.

State-of-the-art Vertical Scroll Sites

http://benthebodyguard.com/ http://2011.beercamp.com/ http://sf-langstrasse.hvo.ch/page/ http://activatedrinks.com/

Official Google Blog: When patents attack Android

Official Google Blog: When patents attack Android: "I have worked in the tech sector for over two decades. Microsoft and Apple have always been at each other’s throats, so when they get into ..."