2012/12/10

Import mails on gmail into PostgreSQL server

I have received many emails from systems. Some are urgent alerts and some are just for my information. I wonder how does the trend of these alerts go so far. It is difficult to analyze the trend just on my mailbox so I  thought that it would be useful to import those mails into a local PostgreSQL server.

First, I enabled POP3 service on my gmail account, then I wrote some ruby code.
With some gems and ruby, I could do these task in handy.
-------------------------------------------------
require 'mail' #gem mail
require 'pg'   #gem pg

Mail.defaults do
  retriever_method :pop3, { 
    :address => "pop.gmail.com",
    :port => 995,
    :user_name => 'username',
    :password => 'password',
    :enable_ssl => true
  }
end

# connect PostgreSQL server
db = PG.connect( :dbname => 'db_name', :user => 'db_user')

Mail.all.each do |email|
  $stderr.puts email.date.to_s # Output date of email
  begin
    # Japanese character sets are often undetected.
    # So I use Content-Type of email.
    if email.content_type =~ /ISO-2022-JP/i then
      body_str = email.body.to_s.encode("UTF-8", "ISO-2022-JP")
    else
      body_str = email.body.encoded
    end
    # insert into PostgreSQL server
    db.exec "INSERT INTO mailbox ( dt, subject, body ) VALUES ($1,$2,$3);", [email.date.to_s, email.subject, body_str]
  rescue => ignore
    $stderr.puts "[error]:" + ignore.to_s
  end
end
-------------------------------------------------

And this code needs two gems, pg and mail.
Those are installed in following commands.
-------------------------------------------------
(For Windows)
gem install --no-ri --no-rdoc pg -- --with-pg-config="c:\Program Files\PostgreSQL\9.2\bin\pg_config.exe"
gem install --no-ri --no-rdoc mail
-------------------------------------------------
(For Linux)
sudo gem install pg -- --with-pg-config=/usr/pgsql-9.2/bin/pg_config
sudo gem install mail
-------------------------------------------------

Now I have been importing mails. When it is finished, it is time to analyze. That is another story.

2012/12/06

Browser test with Selenium and WebDriver

I recently try to make browser tests with Selenium and WebDriver. I found it is very useful and fan. I use to use Selenium-RC and PHP bindings and Ruby bindings. In my project, I use Selenium2, WebDriver and Java bindings.
It is easy to install with maven(pom.xml).

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.10</version>
      <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>2.25.0</version>
    </dependency>
  </dependencies>
And I downloaded Selenium Server at Selenium site.
If you want to use Chrome for your test, Chromedriver is also needed. It could download from this site. Running Selenium Server with Cromedriver is a following command options:

java -jar selenium-server-standalone-2.25.0.jar -Dwebdriver.chrome.driver=C:\path\to\chromedriver.exe

Selenium server is now ready for accept the command from test cases. If you have enough network allowances, The server can accept from even other hosts so that you can make CI server and distributed testing clients.

2012/12/05

FedEx scam email

I received email which suggests I have an undelivered package at a postal office.

I wondered why I receive this notice because I haven't bought any items recently from oversea and FedEx is not a major delivery service in Japan domestically. Then, I found FedEx alert message on the website. It is one of the infamous scam email. The email which I received did not attached any malicious files, but I need to be careful about this kind of emails.

2012/11/30

Useful shortcut keys in Eclipse

I have been learning Java language. Java is one of the popular static language. Static languages have a rigid interface on many aspects. Unlike dynamic languages, like Javasctipt, Ruby and Perl, this rigidness decrease misunderstandings on a complicated system. And we are able to take advantage of this characteristics by using IDE (Integrated Development Environment).
I have been using Eclipse Juno and found that it makes me more productive when I use shortcut keys.
Following are some of the shortcut keys:

Java Editing

1. Content Assist (Ctrl + Space)
2. Quick Assist (Ctrl + 2)
3. Organize Imports (Ctrl + Shift + O)
4. Format (Ctrl + Shift + F)
5. Toggle comment (Ctrl + 7)
6. Quick Fix (Ctrl + 1)
7. Move line up or down (Alt + Arrow)
8. Delete line (Ctrl + D)
9. Quick menu (Alt + Shift + S)

Find
1. Open call hierarchy (Ctrl + Alt + H)
2. Open declaration (Ctrl + Click)
3. Open Type (Ctrl + Shift + T)
4. Open Resource (Ctrl + Shift + R)
5. Go to Line (Ctrl + L)
6. Quick Outline (Ctrl + O)
7. Incremental Search (Ctrl + J)
8. Search (Ctrl + H)
9. Find/Replace (Ctrl + F)

Workspace navigation
1. Close Editor (Ctrl + W)
2. Switch Editor (Ctrl + E)

These are a part of huge amount of shortcut keys in Eclipse. Remember it and have a happy coding time.




2012/11/28

PC server ML110 G7 with VMware ESXi 5.1

After I wrote a previous post on my blog, I received a coupon which had a discount on PC servers. The coupon gave me a 4,000 yen ($50) discount on ML110 G7 which is normally priced at 13,980 yen ($175). I decided to buy it soon and I had started to collect some parts to upgrade the machine.
Finally, I bought following items:

---------------------------------------------
ML110 G7    9,980 yen
WD 2.0TB WD20EZRX/N (HDD)    6,778 yen
Kingston DDR3 KVR1333D3E9SK2/8G (Memory)  5,164 yen x 2
BUFFALO RUF2-PS2GS-BK (USB stick memory)  891 yen
---------------------------------------------
Total       27,977 yen ($350)

I put memories and the HDD into the server when all items had arrived. A body of the server is quite accessible to put items on easily. Furthermore, it has an internal USB port so that I can put the USB stick memory which was installed VMware ESXi 5.1 on it.
The procedure of installing ESXi is following: 
  • Download VMware ESXi 5.1(ISO image) into my pc.
  • Make a bootable USB memory key using unetbootin (It's very useful software).
  • Set the USB memory key into an internal USB port of the server.
  • Set power management setting of the server to "OS Control" in BIOS settings screen.
  • Boot from USB memory key then ESXi installation starts.
  • Install ESXi into USB memory key itself.
Installation is smooth and I got a virtualized server quickly and easily. Finally, I installed VMware vSphere Client on my pc to control the server. At the time, ESXi is installed as evaluation edition which is used only for 60 days so that I needed to input license code, which was provided on VMware's website freely, into the server by using vSphere Client.

2012/11/22

Low price PC server in Japan

I have been looking for a small PC server recently. To be honest, I prefer using cloud services to buying hardware in my local environment, but some restrictions force me to do so.

There are several ways to buy small PC servers. Going to Akihabara, where is the most popular electric town in Japan or visiting web shops offering special sales. Sometimes these shops offer products cheaper than real shops and even maker's official direct shops.

I like a shop called Nttx. The store always sell PC servers for personal use or small sized company use and often offers very lower prices for PC servers. The price is around 10,000 yen($125). I haven't expected that we can get a PC server for such a low price.
Why can they sell a PC server at so low price? The reason is around sales share competitions. Many PC makers in Japan compete each other by taken the number of share percentage. If they lose the number, they seems to be retreating in domestic market so that many sells low cost PC server on slightly margin to get share percentages. Sales are often held in the end of financial term, March, June, September and December.

I wanted to buy a PC server this month but there are no special offer right now so I hesitate to buy it now, maybe I will wait special offers until next month.

2012/11/21

Why there are few hackers in Japan

In this few weeks, many Japanese media insists that Japanese both private and public organisations should educate young people to become hackers for protecting their organization against to cyber-terrorism.

On the background, many attacks to Japanese site are exposed recently and we found there are a very few people who can deal with that. Especially, when Japanese police have done several mistaken arrests this year because of misunderstanding illegal access caused by virus software and CSRF(Cross Site Request Forgery), many people disappointed about public organization's acknowledge level of information technology.

I think main reason of luck of hackers(I don't like this usage in this context, anyway) is stiffness of Japanese labor environment, Japanese High tech people tend to stay in one company for a long time as compared other countries, like U.S. or China. There are few people to build their career by moving across public sector and private sector. Farther more there are few to study at high educational organizations when they became adults. Those are all caused by stiffness of Japanese labor market.

Once you drop the rails of hiring, you couldn't recover that. Ordinarily people who work at a big company think that feeling in greater or less. That allows uneven distribution of knowledge about high technology. Eventually, uneven knowledge bring our society inefficiency and unfairness. Those accidents remind me that it is a good time to change the rules regulating Japanese labor market to ease the stiffness.


2012/11/20

Kindle has finally come in Japan


I received kindle Paperwhite e-book reader yesterday.
I have used some items and bought some books at Japanese Amazon store. Comparing previous Kindle touch, Paperwhite have a brighter screen and little more smooth move in touch.

I have been worry about my account, because I have an account at both Amazon.com and Amazon.co.jp. I was not sure to get my books bought earlier at Amazon.com with storing my items bought at Amazon.co.jp.
As a result, I could have both items. First my kindle was registered by my account at Amazon.co.jp but I could unregister and re-register with my Amazon.com account by switching kindle's system language.
unregister JP account -> swithing system language -> reboot -> register US account (Vice versa)
Some Japanese books are offered by a bargain price at Amazon.co.jp. Some are sales at about 100 yen($1.2). Some are sales at little cheaper price then real books.



I bought some books and comic at Amazon.co.jp.



The screen is brighter than ever model so that I could read easily even small fonts in comic and novel. I have not tried to read for many hours, I could not tell about tiredness on eyes.But I feel good so far. Some parts of UI is defferent from previous Kindle touch. I, however, was not at a loss to use it.

Japanese Amazon store have not only Japanese books but also English books. It is easy to buy English books with Japanese yen and Amazon.co.jp account. That might lower a wall to reach to English books for many Japanese consumers.

2012/11/19

How to get latest PHP and MySQL on VPS server.

Recently I have used a cheep VPS service in Japan. Many of them offer bare bone Linux distribution with minimum software. That is fine, but I got considerably old version of software when I installed PHP or MySQL with yum command. It is not only usefulness but also including vulnerability of whole system.One solution is building from source code and another is using third party repository to install them.

"Les RPM de Remi" is a famous repository for PHP community. If you want to get newest version of PHP and MySQL, you can get them easily by using this repository. Here is the procedure of adding Remi repository to your system (CentOS 6, Other systems are instructed in Remi's site).

# wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm
# wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
# rpm -Uvh remi-release-6*.rpm epel-release-6*.rpm

Then, you can see a version of software that would be installed using info command and install them using install command.

# yum --enablerepo=remi info php
# yum --enablerepo=remi install php php-devel php-mbstring

If you want to install father new version, you can try remi-test repository.

# yum --enablerepo=remi-test info php
# yum --enablerepo=remi-test install php php-devel php-mbstring

As of November 19, we can get PHP5.3.18 and PHP5.4.8 with remi and remi-test respectively. But Remi just announced that remi will change the supporting version of software on December 15.
------
I've decided to stop support of PHP 5.3 in remi repository. So, this is PHP 5.4 (currently in remi-test) which will go in remi.In the remi-test repository, which will get back its role, I will soon propose:
PHP 5.5.0 (alpha should be published really soon)
MySQL 5.6 (version 5.6.8 is a RC).
Planed switch date : december 15th.
------
So if you need to use Remi repository next month, take care of their versions.

2012/11/16

Japanese politics and our hard tough road

Today Japanese diet will be dissolved and next election will be held in December 16. Before DPJ rule current diet, almost all ages, LDP had been ruling Japanese politics after WW2. At the previous election, many Japanese, including me, think that breaking super long term regime brings solutions to our tremendous problems, such as economic, labor and aging society. Now some are better and some are worse. We found only blaming politicians and change prime minister does not work any more.

Here is Murakami Ryu's 2009 Op-ed:
The days when everything worked like a dream and everyone’s standard of living kept rising are over, and have been for a long time. Now that there is no longer enough money, the Japanese public has to make some hard choices.
We are on the middle of hard and tough road. We still have a lot many hard choices now.  In this hard reality, I think I should return to the simple basic, get things done one by one.


2012/11/15

"Biz Dev" minds at start-up company

I read a article "Why we have never hired anyone from the NBA" yesterday and I knew the word "Biz Dev". I think that it is a derivation word of "DevOps." and "Biz Dev" is a good expression to show situations of venture companies. There is no development member who work for only developing. They need to make everything from a plan of business to demonstrate their product.
I have been working at a venture company and I feel "Biz Dev" minds are essential part of developers in the company.

From article,
(1) embracing new technologies and learning to shoot hoops – even if you will never be a tall guy, a basic understanding will deliver a step-change in your effectiveness;
(2) a proven ability to get shit done – start and keep up a squad, blog (hypocritical I know given my recent hiatus!) or other small team; or
(3) knowledge and experience of working at a startup – the only way to really learn about doing a startup is by working at or starting one yourself. (The next best thing is to be well-read on the space and with the number of great resourcesout there you really have no excuse.)
 Each items are good point, we need to learn and act much more faster than a developer who work at a big company. Everyday we are in a classroom at a front line of business competitions.

2012/11/14

Computer Virus targeted Japanese banking site users


October 30, Japanese bank Sumitomo Mitsui banking corporation announced that the bank have attacked by proofing username and password which had stolen by a computer virus.
If a user have affected the computer virus, the virus show the fake input screen when the user access a banking web site.
Here is a video demonstrated by SecureBrain.

Same kind of attacks are reported on Mizuho Bank, Bank of Tokyo-Mitsubishi UFJ, Rakuten bank, SBI Sumishin Netbank, Japan Post Bank, Mitsubishi UFJ NICOS and Mitsubishi UFJ Morgan Stanley Securities.
As of November 12, Illegal remittance are more than 4 million yen ($50,000) and Illegal cash withdrawal are about 1.7 million yen ($20,000).
There is no other virus like this so far, but it is reasonable to think this kind of attack will happen again. It is important to protect your PC from a virus and make sure to input your data into proper website when you access a banking site.

2012/11/13

Amazon Kindle Paperwhite Vs. The Kobo Glo


I received E-mail from Amazon.co.jp. E-mail said that “Kindle Paperwhite” is price down before shipping. It goes down from 8,480 Yen to 7,980 Yen. Amazon don’t notice why they put down the price but clearly Rakuten “kobo glo”, competitor’s new e-book reader, have announced in November 1.
I found a movie comparing two readers at Youtube. That’s interesting.

And, here is a comparative table in Japan.

Rakuten kobo glo
Amazon Kindle Whitepaper
Size
114 x 157 x 10 mm
117 x 169 x 9.1 mm
Weight
185 g
213 g
Processor
1 GHz
Unknown
Display
6” E Ink XGA Pearl screen
6” Paperwhite display
Resolution
1024 x 768
1024 x 768
Screen Color
16-level grey scale
16-level gray scale
Light
Built-in on screen
Built-in on screen
Buttons
Power on/off, light on/off
Power on/off
Connectivity
Wi-Fi 802 b/g/n
Wi-Fi 802 b/g/n
Port
Micro USB
Micro USB
Storage
2GB
2GB
Card slot
Micro SD card (Up to 32GB)
None
Battery
1 month
8 weeks
Surface color
4 colors of back
1 color
Price in Japan
7,980 Yen
7,980 Yen
Release date in Japan
2012-11-15
2012-11-19

As I have seen this table, they are competing products. Which reader will be dominate the market of e-books in Japan? 

2012/11/07

Build error of msysGit on Windows 8


If we use git on Windows environment, many people use Git for Windows.But recently I tried to msysGit instead of that and stuck with installation.

I tried to do it several times. But every time I found error in the middle way. Here is error message.


It seems to success compiling the source code because I found binary files in git directory. However I feel uncomfortable so that I search around web sites.

Fortunately, I found a simple solution at Googlegroups.I could install msysgit after I have set “LANG=C” in OS’s environmental variables. In windows 8, I found that on “Advanced system settings” in System control panel.

What is “LANG”? According to Oracle’s site, it regulates locale of the system. “C” means to POSIX, which includes ANSI C. I can also use “ja_JP.UTF-8” to set Japanese environment. If I have Japanese locale on Japanese Windows, I might have no problem like that.

With Git for Windows, I did not have any problem. If you don't need functionality for developer of git software itself, it is better to use Git for Windows.

2012/01/29

テストページ

単なるテストページです。
うまくいくでしょうか。