2012/12/10

Import mails on gmail into PostgreSQL server

I have received many emails from systems. Some are urgent alerts and some are just for my information. I wonder how does the trend of these alerts go so far. It is difficult to analyze the trend just on my mailbox so I  thought that it would be useful to import those mails into a local PostgreSQL server.

First, I enabled POP3 service on my gmail account, then I wrote some ruby code.
With some gems and ruby, I could do these task in handy.
-------------------------------------------------
require 'mail' #gem mail
require 'pg'   #gem pg

Mail.defaults do
  retriever_method :pop3, { 
    :address => "pop.gmail.com",
    :port => 995,
    :user_name => 'username',
    :password => 'password',
    :enable_ssl => true
  }
end

# connect PostgreSQL server
db = PG.connect( :dbname => 'db_name', :user => 'db_user')

Mail.all.each do |email|
  $stderr.puts email.date.to_s # Output date of email
  begin
    # Japanese character sets are often undetected.
    # So I use Content-Type of email.
    if email.content_type =~ /ISO-2022-JP/i then
      body_str = email.body.to_s.encode("UTF-8", "ISO-2022-JP")
    else
      body_str = email.body.encoded
    end
    # insert into PostgreSQL server
    db.exec "INSERT INTO mailbox ( dt, subject, body ) VALUES ($1,$2,$3);", [email.date.to_s, email.subject, body_str]
  rescue => ignore
    $stderr.puts "[error]:" + ignore.to_s
  end
end
-------------------------------------------------

And this code needs two gems, pg and mail.
Those are installed in following commands.
-------------------------------------------------
(For Windows)
gem install --no-ri --no-rdoc pg -- --with-pg-config="c:\Program Files\PostgreSQL\9.2\bin\pg_config.exe"
gem install --no-ri --no-rdoc mail
-------------------------------------------------
(For Linux)
sudo gem install pg -- --with-pg-config=/usr/pgsql-9.2/bin/pg_config
sudo gem install mail
-------------------------------------------------

Now I have been importing mails. When it is finished, it is time to analyze. That is another story.

2012/12/06

Browser test with Selenium and WebDriver

I recently try to make browser tests with Selenium and WebDriver. I found it is very useful and fan. I use to use Selenium-RC and PHP bindings and Ruby bindings. In my project, I use Selenium2, WebDriver and Java bindings.
It is easy to install with maven(pom.xml).

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.10</version>
      <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>2.25.0</version>
    </dependency>
  </dependencies>
And I downloaded Selenium Server at Selenium site.
If you want to use Chrome for your test, Chromedriver is also needed. It could download from this site. Running Selenium Server with Cromedriver is a following command options:

java -jar selenium-server-standalone-2.25.0.jar -Dwebdriver.chrome.driver=C:\path\to\chromedriver.exe

Selenium server is now ready for accept the command from test cases. If you have enough network allowances, The server can accept from even other hosts so that you can make CI server and distributed testing clients.

2012/12/05

FedEx scam email

I received email which suggests I have an undelivered package at a postal office.

I wondered why I receive this notice because I haven't bought any items recently from oversea and FedEx is not a major delivery service in Japan domestically. Then, I found FedEx alert message on the website. It is one of the infamous scam email. The email which I received did not attached any malicious files, but I need to be careful about this kind of emails.

2012/11/30

Useful shortcut keys in Eclipse

I have been learning Java language. Java is one of the popular static language. Static languages have a rigid interface on many aspects. Unlike dynamic languages, like Javasctipt, Ruby and Perl, this rigidness decrease misunderstandings on a complicated system. And we are able to take advantage of this characteristics by using IDE (Integrated Development Environment).
I have been using Eclipse Juno and found that it makes me more productive when I use shortcut keys.
Following are some of the shortcut keys:

Java Editing

1. Content Assist (Ctrl + Space)
2. Quick Assist (Ctrl + 2)
3. Organize Imports (Ctrl + Shift + O)
4. Format (Ctrl + Shift + F)
5. Toggle comment (Ctrl + 7)
6. Quick Fix (Ctrl + 1)
7. Move line up or down (Alt + Arrow)
8. Delete line (Ctrl + D)
9. Quick menu (Alt + Shift + S)

Find
1. Open call hierarchy (Ctrl + Alt + H)
2. Open declaration (Ctrl + Click)
3. Open Type (Ctrl + Shift + T)
4. Open Resource (Ctrl + Shift + R)
5. Go to Line (Ctrl + L)
6. Quick Outline (Ctrl + O)
7. Incremental Search (Ctrl + J)
8. Search (Ctrl + H)
9. Find/Replace (Ctrl + F)

Workspace navigation
1. Close Editor (Ctrl + W)
2. Switch Editor (Ctrl + E)

These are a part of huge amount of shortcut keys in Eclipse. Remember it and have a happy coding time.




2012/11/28

PC server ML110 G7 with VMware ESXi 5.1

After I wrote a previous post on my blog, I received a coupon which had a discount on PC servers. The coupon gave me a 4,000 yen ($50) discount on ML110 G7 which is normally priced at 13,980 yen ($175). I decided to buy it soon and I had started to collect some parts to upgrade the machine.
Finally, I bought following items:

---------------------------------------------
ML110 G7    9,980 yen
WD 2.0TB WD20EZRX/N (HDD)    6,778 yen
Kingston DDR3 KVR1333D3E9SK2/8G (Memory)  5,164 yen x 2
BUFFALO RUF2-PS2GS-BK (USB stick memory)  891 yen
---------------------------------------------
Total       27,977 yen ($350)

I put memories and the HDD into the server when all items had arrived. A body of the server is quite accessible to put items on easily. Furthermore, it has an internal USB port so that I can put the USB stick memory which was installed VMware ESXi 5.1 on it.
The procedure of installing ESXi is following: 
  • Download VMware ESXi 5.1(ISO image) into my pc.
  • Make a bootable USB memory key using unetbootin (It's very useful software).
  • Set the USB memory key into an internal USB port of the server.
  • Set power management setting of the server to "OS Control" in BIOS settings screen.
  • Boot from USB memory key then ESXi installation starts.
  • Install ESXi into USB memory key itself.
Installation is smooth and I got a virtualized server quickly and easily. Finally, I installed VMware vSphere Client on my pc to control the server. At the time, ESXi is installed as evaluation edition which is used only for 60 days so that I needed to input license code, which was provided on VMware's website freely, into the server by using vSphere Client.

2012/11/22

Low price PC server in Japan

I have been looking for a small PC server recently. To be honest, I prefer using cloud services to buying hardware in my local environment, but some restrictions force me to do so.

There are several ways to buy small PC servers. Going to Akihabara, where is the most popular electric town in Japan or visiting web shops offering special sales. Sometimes these shops offer products cheaper than real shops and even maker's official direct shops.

I like a shop called Nttx. The store always sell PC servers for personal use or small sized company use and often offers very lower prices for PC servers. The price is around 10,000 yen($125). I haven't expected that we can get a PC server for such a low price.
Why can they sell a PC server at so low price? The reason is around sales share competitions. Many PC makers in Japan compete each other by taken the number of share percentage. If they lose the number, they seems to be retreating in domestic market so that many sells low cost PC server on slightly margin to get share percentages. Sales are often held in the end of financial term, March, June, September and December.

I wanted to buy a PC server this month but there are no special offer right now so I hesitate to buy it now, maybe I will wait special offers until next month.

2012/11/21

Why there are few hackers in Japan

In this few weeks, many Japanese media insists that Japanese both private and public organisations should educate young people to become hackers for protecting their organization against to cyber-terrorism.

On the background, many attacks to Japanese site are exposed recently and we found there are a very few people who can deal with that. Especially, when Japanese police have done several mistaken arrests this year because of misunderstanding illegal access caused by virus software and CSRF(Cross Site Request Forgery), many people disappointed about public organization's acknowledge level of information technology.

I think main reason of luck of hackers(I don't like this usage in this context, anyway) is stiffness of Japanese labor environment, Japanese High tech people tend to stay in one company for a long time as compared other countries, like U.S. or China. There are few people to build their career by moving across public sector and private sector. Farther more there are few to study at high educational organizations when they became adults. Those are all caused by stiffness of Japanese labor market.

Once you drop the rails of hiring, you couldn't recover that. Ordinarily people who work at a big company think that feeling in greater or less. That allows uneven distribution of knowledge about high technology. Eventually, uneven knowledge bring our society inefficiency and unfairness. Those accidents remind me that it is a good time to change the rules regulating Japanese labor market to ease the stiffness.