想自建vpn，就必须有vpn的服务器。这种服务器一般是利用云服务公司提供的虚拟机服务VPS（Virtual Private Server）。业界比较知名的供应商有Amazon EC2，Microsoft Azure，Linode，DigitalOcean等等。这几个应当都可以；如果是单纯做vpn翻墙供自己或小团队使用，供应商的最小配置就可以。我现在用的就是单CPU、128MB内存、10G硬盘小虚机，够我们多台设备一起用；挑选数据中心时，不建议用英语国家的数据中心，特别是美国的。一是有安全和隐私方面的担心，相信大家都读过相关新闻；二是至少我尝试过的美国数据中心访问中国的一些网站都慢的要死，如优酷。但我发现西欧的数据中心如德国、荷兰、瑞典就没有这方面的问题。我不知道为什么。操作系统当然要用Linux。我建议Debian。接下来的how to我会假定你的操作系统是Debian 7或8。如果你用其它操作系统，接下来的how to仍然会有帮助。先写到这儿。亲爱的读者，你先去研究下哪个服务更适合你。之后再回来看我的指南。接下来的博客会提供些建议，讲如何提高你的虚拟机的安全。然后我们再说建vpn的事儿。 ...继续阅读 (47)

For my own reference. Hope it’s helpful to you as well.Given this text pattern:Keyword blah blah keyword blahLine I don’t care.Keyword blah blah keyword blahI want everything after this line.Good stuffGood stuffRecipe 1: Returning all lines with keyword://string pattern = @"keyword .+ keyword .+"; public string GetKeywordLines(string pattern, string inputText) { StringBuilder sb = new StringBuilder(); foreach (Match m in Regex.Matches(inputText, pattern)) { sb.AppendLine(m.Groups[1].Value); } ...继续阅读 (45)

I wrote ascript to generate script for non-clustered index page compression in SQL Server. I’ve since made improvements and here it is.Highlights:This script generates index compression script on a partition by partition basis, if the underlying index is partitioned;Like the previous version, it still works for non-partitioned indexes;Like the previous version, it compresses indexes from smallest to largest, progressively saving space as it nibbles forward. Therefore it is unlikely that it’ll grow data file(s) during rebuild;Unlike the previous version, I’ve set maxdop = 0, w ...继续阅读 (52)

One of the best investments I’ve ever made is to be proficient with a good cross-platform editor, in my case Vim. It took me a good few months before I really became comfortable with it, but those few months’ struggle yielded huge dividend since then!So after years of Vim usage, I consider myself a power user. Yet from time to time, I come across some nifty tips that remind me why I fall in love with it in the first place: the sense of wonder, awe, beauty, and intelligence of its creators!Here are two things I learned recently:Sort based on a patternI use sort and sort u all the ti ...继续阅读 (47)

视频演示：1. 怎么更新Ubuntu Linux；sudo apt-get updatesudo apt-get upgradesudo apt-get install build-essential2. 怎么安装Python包；sudo apt-get install python-pipsudo apt-get install python-devsudo pip install numpysudo pip install ggplot…3. 怎么安装R和R包；sudo apt-get install r-basesudo apt-get install openjdk-7-jdksudo Rinstall.packages(“xlsx”)4. 如何方便快捷下载视频:Firefox, 插件DownThemAll ...继续阅读 (32)

最近在几个QQ和微信IT群里灌水，注意到不少同学在大学或工作中没接触过Linux。而很多IT项目如大数据、机器学习、服务器等都需要Linux技能，所以很多人想开始接触、学习Linux。我就动手做了以下视频给初学者，希望能有帮助。这是我第一次做screencast，很希望能听到你的批评和建议。更新：视频上传到优酷后，效果并不理想。我又尝试了其它视频分享网站如乐视、QQ视频、新浪视频、和土豆。乐视的上传网页没有上传渠道，或许因为我的IP地址在国外？我在新浪视频网页也碰到同样问题。QQ视频倒是允许上传，但最终告诉我“您的视频可能包含有相关主管机关明确规定不能出现的违规内容，因此无法通过审核。请修改后再重新上传。”，真是令人匪夷所思。最后上传到土豆，效果还可以。我是把几个小视频合并到一个文件，但由于操作错误，我没有把一开始介绍VirtualBox软件的那部分合并进去。你只要记得VirtualBox在Windows、Linux、Mac上都可以免费运行并去下载安装就可以啦。 ...继续阅读 (45)

It seems the venerable file format CSV (Character/Comma-separated values) never goes out of style. According toCSV wikipedia page, IBM Fortran started supporting this format in 1967, before many of us were born. CSV has been with us, through thick and thin, silently but steadfastly, ready to spring into action when duty calls. For sure, it’s one of data professionals’ best friends! Often times, we’d convert spreadsheet files or dump data inside a database into a CSV before it can be distributed and consumed downstream.Major league scripting languages, such as Perl, Python, an ...继续阅读 (29)

I had trouble rsync remote Linux 600 files (rw——-) today. I knew that I came across this issue before but couldn’t remember how I resolved it. Therefore I had to waste time looking for and verifying a solution. Hence this blog post.This is the problem I had earlier:rsync -zr userA@remoteServer:/var/www/website/ /home/user/Documents/webSiteBackup/website/www/ rsync: send_files failed to open "/var/www/website/wp-config.php": Permission denied (13) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [generator=3.1.0]So the issue is ...继续阅读 (45)

For a Windows application or service, it is often necessary for either the user or service startup account to have full control to its folders and files. In most cases, this is transparent to the user. It is generally being taken care of by various delegation mechanisms behind the scene. However, this can be a problem during application upgrade or migration, when one rearranges disk drive letters or mount point names, or moves things from one server to another. To resolve this, one could manually go to each file and folder property page through Windows Explorer, then the Security tab, then the ...继续阅读 (44)

I helped a charity to rebuild a MySQL server and to restore a database with a lot of data of longblob type in the last two days. Fortunately there was a dump backup file for the database in question.However, tables with longblob column(s) were not defined with “ROW_FORMAT=COMPRESSED”. I’d like to restore that database with row compression before inserting the data. Therefore I need to modify the dump sql file. The problem is that the file is 2.5 GB and the server only has 4 GB memory. So editing it is a challenge. Fortunately, Linux has sed to save the day. Don’t you lo ...继续阅读 (25)

For BI data warehouse databases, since the data does not change much and they typically require a lot of space, it makes a lot of sense to compress the indexes to save space.I came across some BI databases whose indexes were created without compression. We are in the process of migrating those databases to a new server so I took this opportunity to completely rebuild those indexes with PAGE compression.Two things are of interest:Index rebuild starts from the smallest to the largest. The rational is that during one index rebuild, it needs roughly twice of the actual index size. If we start with ...继续阅读 (37)

最近读到老家山东省畜牧局在养殖业发展沼气利用的新闻，让我很受鼓舞。我就把最近在网上读到的公开资料在这里汇总一下，希望能有帮助。也提出一些想法和问题，算抛砖引玉。这里是欧盟和几个波罗的海国家合作搞的BalticBiogasBus项目的网站。该项目目的是利用废弃有机物生产的沼气经提升（提高甲烷含量）后给公交系统提供燃料。由于甲烷燃料的排放仅仅是二氧化碳，比汽柴油排放清洁太多，对改善环境、治理雾霾有极大帮助。在页面PUBLICATIONS栏目下有非常多好文章，很值得借鉴。比如“在公交系统引进沼气/天然气公车的策略、政策、和行动计划指南”。又如“沼气生产经验和教训”等等。沼气给公交车提供燃料时其甲烷CH4含量需超过90%。这篇文章是沼气升级的概述，可以和下面Biogasmax项目文章提到沼气升级文章对照来看。另外，在该网站的NEWS栏目下也有很多好文章，比如这篇就讲到瑞典南部城市马尔默计划利用沼气给研发中的快速公交系统BRT提供燃料。斯德哥尔摩的大沼气公交：这里是瑞典Biogasportalen的关于Biogas的比较详尽的介绍，英文版；这里是欧盟几年前的Biogasmax项目网站，有很多公开的科研成果，很有意思。比如这个pdf文章就讲到给沼气升级的几种途径（water scrubbing等）。该网站列出的其它文章也很有意思；美国加州大学戴维斯分校的张瑞红教授是厌氧沼气利用专家。美国环保部 ...继续阅读 (30)

我记得最初学到沼气是在初三的化学课，课本里讲到一些有机物在缺氧的条件下发酵分解，生成主要成分为甲烷（CH4）的气体。英文称沼气为methane。近几年来biogas这个词开始流行，这倒不是说在我们每个人屁股后装个袋子采集来自大肠的气体，不是那个意思，请大家不要误解。我们可以通过下面这张图表来简要介绍下biogas的生产和使用过程。该图介绍了怎样生产沼气。还有一种在垃圾填埋场（landfill）采集沼气的方法，下文也会提到。上图中心的圆柱拱形建筑是沼气生产的核心部件，叫消化器（digester，或anaerobic digester）。该容器内无光无氧，是利用微生物发酵分解原料的地方。原料厌氧发酵后产生的沼气会飘到拱形的顶部被采集。一般消化器里会有搅拌装置，来促进有机物的厌氧发酵过程。这是位于德国的Windwärts Energie GmbH的消化器建设中的图片：这是同一家biogas能源工厂的消化器顶层照。请注意其中的木材使用。据介绍，屋顶下的木质结构是用来减少沼气中的硫化物。太多硫化物对CHP热电机组不利：消化器的侧面会有原料入口和余料出口。原料（substrate）如上篇博客所说，可以是很多种废料：农作物废料如秸秆，城市绿化废料如废弃枝叶杂草，餐饮和食品加工废料如剩饭菜和上海福喜的过期麦乐鸡或小牛扒，养殖场动物粪便，能源农业生产的农作物，城市污水系统里的污泥（人的粪便）等等。除 ...继续阅读 (28)