Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 947

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/samsongli/meirendaddy.com/blog/wp-includes/kses.php on line 948
April 6, 2012

今天在微博上面看到  @邓侃 邀请大家谈谈关于京东Site Is Too Busy,原文在这里:

http://i.wshang.com/?p=4247

我在云计算方面不能算大牛,说不出什么惊世骇俗的话来,也没有指点江山的豪气。这里凭着在业界做了这么多年实战经验,说点实在的:

  1. 我们做server的,在实现之前,几个事情要做:设计,把任务分割好,静态内容和需要计算的分开,尽量减少对系统的压力(CPU,内存,网络带宽)。
  2. 实现中间一定要注重logging,一定要。程序员不愿意写logging代码,每少一条logging,就是一个丢钱的漏洞,要以后付出巨大代价来弥补。
  3. 系统上线之前,压力测试(stress test)必须做,认真做,尽量真实。对问题,早发现早解决。
  4. 上面三条是老生常谈,让人听了烦。但是能做到的,就不会出大问题。我当年做的东西,全欧洲的人查酒店都用,现在扩展到亚太,一样好好的,就是几台机器而已。因为设计上面瓶颈都被分散了。
  5. 关于增加服务器来缓解压力。这个如果设计上面做的好,应该可以有很大帮助。但是如果瓶颈不能用增加服务器的方式来分散化解,增加的服务器只是在那里耗电,没有用。
  6. SQL Server还是Oracle?这俩作为data storage,我以前做过的系统里面都用过。Expedia用的就是SQL Server,也活的好好的。
  7. Oracle是不是神谕,看你怎么用。我做过的一个系统,后端是Oracle,全美几百万用户天天用,服务器那里每个端口每秒500次访问,作为中间层的tomcat闲得发慌,Oracle差点趴下了。为神马?因为设计的问题,当然更多的是公司政治,不提。
  8. 多扯一句:NOSQL是不是万能药?美国人说,没有银弹(没有万能药)。用得好,王八拳也是必杀技,你可以用MongoDB去架个静态内容的服务器,但是你要是用MongoDB去处理logging的部分,就比较2,因为MongoDB的写入时间没保障。
  9. 回到logging。log不是用来玩的,要分析,找出错点,找性能瓶颈。
  10. 我对京东的架构的了解,来自上面给出的文章。如果有哪位行内的哥们姐们愿意分享更多细节,我也愿意多唠叨点意见,跟大家交流。或者在微博上面  @美人她爹 也行。另外,我这个博客好久没打扫,评论里面灰尘很多,正在清理中。
  11. 想起一点:设计系统的时候,记住:系统是要出问题的,是要被更新的。出问题的时候要减少损失,更新的时候尽量平稳过渡。如果以后再想起别的再添上。
December 31, 2010

嗯,马上要新年了,顺着解剖 Facebook 【6】云计算云存储的那些事儿 写点关于amazon dynamo的段子。

《最近比較煩》是周華健、品冠、李宗盛这三个在人生不同阶段的男人对男人辛苦的调侃,那么如果映射到云存储提供公司里面的不同职位的人,是否对云存储也有不同的感触呢?

硬件工程师:最近比较烦比较烦比较烦,amazon的机器已经有几十万,平时沉稳淡定的硬盘,每天趴窝也成了家常便饭。

软件架构师:最近比较烦比较烦比较烦,他们硬件麻烦要我们软件来管,机器趴窝还不算完,一个破路由器把网络分两半。

CTO:我比你烦也比你烦也比你烦,贝索斯说了技术细节他不管,用户选了东西你就给我存好,将来读不出来小心你的饭碗!

合唱:每天总有远的近的麻烦,现成的软件都有点脚软。文件系统搞得东西太乱,数据库总是弄得太慢。干脆,咱们卷起袖子自己干!

硬件工程师:机器趴窝不是那么普遍,每天也就几十个节点

软件架构师:数据丢了可是不能算完,我写它一万遍啊一万遍

硬件工程师:路由器的事情老板不肯出钱,买的都是自家网站卖的低端

软件架构师:低端的路由,把网断开,把数据版本分两段…

硬件工程师:你跑调了,那个是《城里的月光》

软件架构师和硬件工程师望着CTO:老板,怎么办?

嗯,歌唱好了,说说唱的都是是啥吧。其实都是云存储里面的常见问题。首先是硬件稳定性,因为云的计算和存储都是海量的PC机完成的,如果一台机器每天可能失效的几率是万分之一,那么基本上每天都有机器失效,特别是数据中心这种地方,一不小心就过热。对付硬件失效的方法就是复制数据和计算,带来的副作用就是有多个copy需要更新。

其次是网络稳定性,这个和硬件稳定性是相关的,通常是因为路由器的关系,让网络分割(partition)为不连通的两个或者多个部分,导致数据不能流通。网络一旦分割,更新copy的事情就会出问题,在一个网络分割的copy被更新了,另外一些在其他网络分割的就不能被更新。将来网络恢复了,就会发现有多个版本。

如何解决这些问题呢?不同的应用有不同的要求。在amazon的这个应用里面,CEO有两个需求:

  1. 用户存的东西(主要是购物车的内容)要立刻能存,不能丢
  2. 用户取数据的时候要给一个完整的购物车

另外,amazon需要的不是普通的文件系统,也不是关系型数据库,他们需要的是一个能快速存取的key / value store,就是用一个key能够快速的取得相应的数据对象,这决定了他们:

  1. 不需要多层的文件系统,因为这个是给程序用的
  2. 不需要关系型数据库,他们主要是拿了key就能立刻取得value,关系型数据库带来很多冗余和不必要的限制
  3. 需要在各种困难情况下迅速的取得数据

具体的实现细节,在这里:

http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf

更多的段子,请关注ifanr的sunny_kan

October 3, 2009

这个云计算,很成问题啊

July 17, 2009

这里看到的,Amazon从用户的Kindle里面删除了某些电子书,然后退钱给用户。

Techcrunch的报道里面,有一张烧书的照片,让我第一时间想起焚书坑儒这个词。

我想这一定是某些娱乐圈里的人梦寐以求的事情:无论是电影也好,艳照也罢,在用户正在看的时候,忽然跳出一行字:本文件将在5秒后自动销毁… 他们甚至把这个疯狂的想法放到了阿汤哥演Mission Impossible里面。奇怪,娱乐圈里的人,脑子怎么都那么怪?

我想云计算的鼓吹者们一定很郁闷,前面twitter的google doc刚刚被盗,Amazon又把手从云彩里面伸到了用户口袋里。下周我再坐公车的时候,遇到那个看Kindle的女士,一定要问问她是不是知道这件事情。

April 8, 2009

写两句想法。

最近比较忙,好在现在空闲了一下,因为公司的开发环境出了问题,所有依赖这个环境的开发都只好停下来了。我跟同事说,仿佛回到了当年只有一台计算机,一个系的人一起用的时代,机器坏了,大家都抓瞎。

这也是我不看好云计算的原因,如果大家都依赖一个集中的东西,这个集中的东西坏了,所有依赖它的事情都要停下来。这不仅是软件开发或者互联网特有的,在经济,政治甚至机械上面,只要系统里面有一个集中的部件,有很多东西依赖它,这个部件就必须设计的非常可靠。云要花多少时间才能变得可靠,我不知道。

我只知道复杂的系统可靠性会下降,即使加入冗余,也是一样。

February 6, 2009

最近四处看到很多关于云计算的东西。说实在的,我第一次听说这个概念的时候,感觉就好像用搜狗输入法打”yuncai”结果一样:晕菜。

各个大公司的副总裁们到处说,云计算是好多PC连在一起,集群,MapReduce,容错,价格优势,不怕病毒,还有好多好多好处。用户只要拿个上网本,或者拿个智能手机,就走遍天下都不怕。可是我就不明白,从用户这边看过去,这一大片云彩,跟当年的大型机,有啥不一样的呢?

除了以前用的是单色终端,现在用的是彩屏上网本;以前用的是连线的终端,现在用的是无线网;以前用的是命令行,现在用的是web,是图形界面;以前用的是一个大型机,现在是一堆PC在一起。但是,那个在中心的计算工具,依然是一堆硬件和软件,还有用户的数据;用户在上面的权限,依然是被某些管理员控制;用户的数据,依然是可以被管理员偷窥;容错虽然比以前好了,但是出错的时候,用户依然是无能为力。

与其把所有的数据都交给这样一朵大云彩:

我宁可相信这样一堆小云彩:

当年用户从大型机的城堡里冲出来,尽管面对死机,病毒,硬盘失效的危险,但是对自己数据的控制,对计算能力的追求是无价的。如今云计算所宣称的廉价,易用,其实更多的是PC系统应做但没有做的事情。相对于一朵大云彩,我更希望我的车库里有一个小的集群,有64个内核的处理能力,有32TB热备份的硬盘。我只需要在家里用上网本远程桌面登录这个集群上面运行的一个虚拟机,剩余的处理能力去支持家里其他人的需求。如果感染了病毒,就删除这个感染的虚拟机;如果硬盘坏了,买个新的插上;如果机器旧了,买个新盒子插上;机器现在300美刀一个,至少能用两年,我才不要每个月花30刀去玩什么我不能控制的云彩。

我买上网本,绝对不是讨厌PC的计算能力,我只是讨厌巨大的机箱,沉重的电池,发热的硬盘。这些东西可以远在天边的大云彩里,但是如果在手边的小云彩里,岂不是更好?我要那个大云彩干什么?我家里又没有整个的互联网数据要处理,我只要足够我自己用的处理能力就可以了。让那个大云彩去处理整个互联网吧,反正它是免费的。

要拥有家里的小云彩,我需要小规模的集群软件,我需要虚拟机软件。这些都是PC软件商应该做的,也是他们新的机会。让用户继续拥有他们的数据吧,不管这些数据是作业,是账单,是情书,是多年前的记忆,还是藏在18层目录下的A片,它可以是任何用户不愿意让别人看到的东西。既然上次我们从大型机的城堡里逃走的时候,每个人都带了一个大型机的克隆,那么这次我们为什么不能带一片小云彩走呢?

让“挥一挥衣袖,不带走一片云彩”永远的留在诗篇里吧,工程师要做的,是

挥挥扳手,顺走一片云彩,放在安全的储藏室里。