 |
Display image of
|
Sunday 9 December 2012
Betteridge's law of headlines
I dont have much to add here but the Wikipedia article is well worth a read http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines
Thursday 29 November 2012
LaTeX - The IEEE Surveys & Tutorials Way (Pt 2)
This the second article in a series on using the IEEEtrans and LaTeX to form a survey suitable for submission to IEEE surveys & tutorials
Yesterday's article made use of the bare bones template for journals provided by IEEEtrans, now we are going to look at the requirements of a survey in IEEE surveys & tutorials
The focus of this article is a survey not a tutorial, therefore all of the points below are regarding surveys and not necessary tutorials.
IEEE Communications Survey & Tutorials is a free online journal published by the IEEE Communications Society. Article appear in both the IEEE Xplore Electronic Library and the IEEE ComSoc Digital Library
To finding existing surveys to read and use as examples, you can sign up to a guest account with IEEE ComSoc Digital Library which will give you free access, at the time of writting, the IEEE ComSoc Digital Library website was down, so I've not yet been able to verify this.
IEEE Communications Surveys and Tutorials is targeted for the generalist throughout the field of communications and communications networking. Intended readers include those involved in research, development, deployment, or instruction in fields related to communications.
All surveys should be tutorial in nature and should be written in a style comprehensible to readers outside the speciality of the article.
Typically, mathematical equations should be kept to a minimum, and not be used unless they are vital to the presentation. Articles intended to instruct in a mathematical area should be presented at a level comprehensible to readers with appropriate backgrounds, and the appropriate background should be carefully delineated in the introductory section of the article.
The term survey, as applied here, is defined to mean a survey of the literature. A survey article in IEEE Communications Surveys & Tutorials should provide a comprehensive review of developments in a selected area, covering its development from its inception to its current state and beyond, and illustrating its development though liberal citations from the literature. Such citations would naturally lead to an authoritative and comprehensive bibliography. The objective is to provide the reader with a sense of the history, development, and future of the topic area, including its interactions with other areas, and enough information to comprehend the development and identify the major players.
As an example, the article "A survey of markup languages" might discuss a number of markup languages, such as WML, XML, HTML, CHTML, and voiceXML. The article might define the term "markup language" and describe some general features and objectives by way of introduction. The article might then provide a time-line of events leading to the advent of markup languages, citing major milestones and breakthroughs. From there, the article might describe the markup languages in chronological order, showing how previous languages developed from previous ones through liberal citations to the literature. The article might conclude by giving the author's well-thought-out opinions on the future.
References must be numbered sequentially, not alphabetically. The basic reference format is: [#] L. Brakmo and L. Peterson, " TCP Vegas: End to End Congestion Avoidance on a Global Internet," IEEE JSAC, vol. 13, no. 8, Oct. 1995, pp. 1465-80.
Authors must clearly state the category of the article in the abstract and again in the introductory section and also clearly state the scope of the article. For example, there must be a statement of the form "This article surveys the literature over the period 1990-2001 on turbo codes as they apply to wireless communications."
Authors are encouraged to consider inclusion of multimedia materials in cases where such material would substantially improve the value of the article to the reader. HOW ???
Figures and tables should be used liberally.
There are no limits on paper length, number of figures, references, etc.
Required formats for electronic submission is PDF.
Submit survey's using the LaTex structure. Include the compiled pdf, figures (as eps files with fonts embedded), bios and photos in the final article, keywords, and abstract
(Note: For the best presentation of your article's entry in IEEE Xplore, do not include equations in the abstract.)
Once the survey is complete it's submitted via the ManuscriptCentral website
Yesterday's article made use of the bare bones template for journals provided by IEEEtrans, now we are going to look at the requirements of a survey in IEEE surveys & tutorials
IEEE Surveys & Tutorials
The following information is largely taken from the Information for Authors page, on the IEEE Communications Surveys & Tutorials site.The focus of this article is a survey not a tutorial, therefore all of the points below are regarding surveys and not necessary tutorials. IEEE Communications Survey & Tutorials is a free online journal published by the IEEE Communications Society. Article appear in both the IEEE Xplore Electronic Library and the IEEE ComSoc Digital Library
To finding existing surveys to read and use as examples, you can sign up to a guest account with IEEE ComSoc Digital Library which will give you free access, at the time of writting, the IEEE ComSoc Digital Library website was down, so I've not yet been able to verify this.
IEEE Communications Surveys and Tutorials is targeted for the generalist throughout the field of communications and communications networking. Intended readers include those involved in research, development, deployment, or instruction in fields related to communications.
All surveys should be tutorial in nature and should be written in a style comprehensible to readers outside the speciality of the article.
Typically, mathematical equations should be kept to a minimum, and not be used unless they are vital to the presentation. Articles intended to instruct in a mathematical area should be presented at a level comprehensible to readers with appropriate backgrounds, and the appropriate background should be carefully delineated in the introductory section of the article.
The term survey, as applied here, is defined to mean a survey of the literature. A survey article in IEEE Communications Surveys & Tutorials should provide a comprehensive review of developments in a selected area, covering its development from its inception to its current state and beyond, and illustrating its development though liberal citations from the literature. Such citations would naturally lead to an authoritative and comprehensive bibliography. The objective is to provide the reader with a sense of the history, development, and future of the topic area, including its interactions with other areas, and enough information to comprehend the development and identify the major players.
As an example, the article "A survey of markup languages" might discuss a number of markup languages, such as WML, XML, HTML, CHTML, and voiceXML. The article might define the term "markup language" and describe some general features and objectives by way of introduction. The article might then provide a time-line of events leading to the advent of markup languages, citing major milestones and breakthroughs. From there, the article might describe the markup languages in chronological order, showing how previous languages developed from previous ones through liberal citations to the literature. The article might conclude by giving the author's well-thought-out opinions on the future.
References must be numbered sequentially, not alphabetically. The basic reference format is: [#] L. Brakmo and L. Peterson, " TCP Vegas: End to End Congestion Avoidance on a Global Internet," IEEE JSAC, vol. 13, no. 8, Oct. 1995, pp. 1465-80.
Authors must clearly state the category of the article in the abstract and again in the introductory section and also clearly state the scope of the article. For example, there must be a statement of the form "This article surveys the literature over the period 1990-2001 on turbo codes as they apply to wireless communications."
Authors are encouraged to consider inclusion of multimedia materials in cases where such material would substantially improve the value of the article to the reader. HOW ???
Figures and tables should be used liberally.
There are no limits on paper length, number of figures, references, etc.
Required formats for electronic submission is PDF.
Submit survey's using the LaTex structure. Include the compiled pdf, figures (as eps files with fonts embedded), bios and photos in the final article, keywords, and abstract
(Note: For the best presentation of your article's entry in IEEE Xplore, do not include equations in the abstract.)
Once the survey is complete it's submitted via the ManuscriptCentral website
Wednesday 28 November 2012
Ocaml 2 HTML
System programming in OCaml - Part 1
This series of articles will follow my journey through "Unix system programming in OCaml", available as a pdf here. After this series, I hope to move onto a series on Mirage system programming in OCaml and working with Signposts (a framework for managing end-to-end connectivity between devices using DNS infrastructure for signalling)
The modules that give access to the system in OCaml are:
NOTE that Unix and Sys can overwrite some of the functions in the Pervasive module, so to access a function from the Pervasive module you need to add Pervasives. in front of the function call.
When using ocamlc to compile OCaml programs that made use of Unix, you need to pass unix.cma to the compiler e.g.
$ ocamlc unix.cma hello.ml -o hello
The same is not true of sys.cma
Now we are going to consider 2 different ways of passing information from your terminal to your ocaml program
1) Passing the information as arguments to the executable. For this we are going to use Sys.argv which will return a string array of arguments passed to the program. The first index of the argument array that we use here is 1, this is because the 0th element in the array is the command used to call the program. Consider the following simple example:
open Unix;;
let env = environment();;
for i=0 to (Array.length env) -1 do
print_string env.(i);
print_newline);
done;
Don't forget then when you compile this, you need to add unix.cma to the command as we are making use of the Unix library
2b ) We can also lookup the values of environment variables using the Sys.getenv (or Unix.getenv) functions that takes a variable name and returns the value associated with it
open Sys;;
let var = argv.(1);;
print_string (getenv var)
Exception Unix_error of error * string * string
The first argument is the error code, the second argument is the name of the system call that raised the error and third argument identifies the object on which the error was called.
If an exception reaches the top-level then the program will halt. The Unix module provides a function for clearly describing exceptions. We can make 2a use this function as follows:
open Unix;;
let prog () =
let env = environment();;
for i=0 to (Array.length env) -1 do
print_string env.(i);
print_newline);
done;
done;
handle_unix_error prog ();;
Introducing the Sys and Unix
- Sys - functions common to all OS's that run OCaml
- Unix - functions specific to unix
NOTE that Unix and Sys can overwrite some of the functions in the Pervasive module, so to access a function from the Pervasive module you need to add Pervasives. in front of the function call.
When using ocamlc to compile OCaml programs that made use of Unix, you need to pass unix.cma to the compiler e.g.
$ ocamlc unix.cma hello.ml -o hello
The same is not true of sys.cma
PASSING INFORMATION TO YOUR PROGRAM
Now we are going to consider 2 different ways of passing information from your terminal to your ocaml program1) Passing the information as arguments to the executable. For this we are going to use Sys.argv which will return a string array of arguments passed to the program. The first index of the argument array that we use here is 1, this is because the 0th element in the array is the command used to call the program. Consider the following simple example:
open Sys;;
print_string argv.(1)
print_string argv.(1)
Now provided that at least one argument is passed to the program, this argument will be printed. We can now extend this program, to print the number of arguments and then exactly what the user will have just entered
open Sys;;
let args = Array.length argv;;
print_int args ;;
print_newline ();;
for i=0 to args-1 do
print_string argv.(i);
print_char ' ';
done;
2a ) Passing information using the environment variables, this can be fetched from within the OCaml programing using Sys.enviroment, this will return as array of environment variableslet args = Array.length argv;;
print_int args ;;
print_newline ();;
for i=0 to args-1 do
print_string argv.(i);
print_char ' ';
done;
open Unix;;
let env = environment();;
for i=0 to (Array.length env) -1 do
print_string env.(i);
print_newline);
done;
Don't forget then when you compile this, you need to add unix.cma to the command as we are making use of the Unix library
2b ) We can also lookup the values of environment variables using the Sys.getenv (or Unix.getenv) functions that takes a variable name and returns the value associated with it
open Sys;;
let var = argv.(1);;
print_string (getenv var)
Exceptions
Unless otherwise stated in the documentation, all function in the Unix module raise the exception Unix_error.Exception Unix_error of error * string * string
The first argument is the error code, the second argument is the name of the system call that raised the error and third argument identifies the object on which the error was called.
If an exception reaches the top-level then the program will halt. The Unix module provides a function for clearly describing exceptions. We can make 2a use this function as follows:
open Unix;;
let prog () =
let env = environment();;
for i=0 to (Array.length env) -1 do
print_string env.(i);
print_newline);
done;
done;
handle_unix_error prog ();;
LaTeX - The IEEE Surveys & Tutorials Way (Pt 1)
This article aims to give an overview of setting out a survey for submission to an IEEE Journal.
For this article, I will be using ubuntu 12.10, 32 Bit and Vim as my text editor.
IEEEtran is the offical LaTeX class for authors of IEEE transaction journals and coferences
My primary source of information for this article is the latest version of the IEEEtran package IEEEtrans director. Other helpful information is available here.
INSTALLATION & SETUP
The IEEEtrans directory (as linked above) includes a "bare bones example of a IEEE journal, called bare_jrnl.tex. We will now use the IEEEtran LaTeX class to compile this .tex file into a pdf file.
To find out the location that we need to place IEEEtran.cls in, use:
$ locate article.cls
For me, the first location returned is /usr/share/texlive/texmf-dist/tex/latex/base/article.cls
I now need to move the IEEEtran.cls that I download into the same directory as article.cls. So move the the IEEEtrans directory and execute:
$ sudo cp IEEEtran.cls <location-of-article.cls>
You can now tell Tex about this new LaTeX class using:
$ sudo texthash
Latex is already installed on my system but to allow us of the IEEEtran class, you may also need:
$
If you get an error of the form:
For this article, I will be using ubuntu 12.10, 32 Bit and Vim as my text editor.
IEEEtran is the offical LaTeX class for authors of IEEE transaction journals and coferences
My primary source of information for this article is the latest version of the IEEEtran package IEEEtrans director. Other helpful information is available here.
INSTALLATION & SETUP
 |
| The essential book on LaTeX |
To find out the location that we need to place IEEEtran.cls in, use:
$ locate article.cls
For me, the first location returned is /usr/share/texlive/texmf-dist/tex/latex/base/article.cls
I now need to move the IEEEtran.cls that I download into the same directory as article.cls. So move the the IEEEtrans directory and execute:
$ sudo cp IEEEtran.cls <location-of-article.cls>
You can now tell Tex about this new LaTeX class using:
$ sudo texthash
Latex is already installed on my system but to allow us of the IEEEtran class, you may also need:
$
sudo apt-get install texlive-fonts-recommendedIf you get an error of the form:
Font OT1/ptm/m/n/9=ptmr7t at 9.0pt not loadable: Metric (TFM) file not found
Then it is likely that you need to install the texlive-fonts-recommended package.
Now we are going to compile the bare_jrnl.tex, by moving to its directory and excauting:
$ latex bare_jrnl.tex
You can now view the outputted document using:
$ xdvi bare_jrnl
NEXT STEPS
Now you have a bare bones copy of your paper, you can customize it to meet your specific requirements and add the content of your paper. I will look at this in more detail tomorrow. is the Information for Authors
Thursday 8 November 2012
Signposts - The installation (DRAFT)
This is a guide to the installation of Signpost on a 64 bit edition of ubuntu 12.04.
The steps of the installation process are:
1 install OCaml
2 install and set up OPAM
3 use OPAM to install the latest OCaml compiler and switch to this new compiler version
4 Add the remote repositories required for signposts to OPAM
5 Install the libraries required using OPAM
6 Download the code for signpost
7 Set up vswitch
8 Generate and place keys
9 Install iodine and set up password
10 Run Signposts
1 INSTALLING OCAML
OCaml 3.12.1 is available from the ubuntu repositories, this is not a up to date version of the compiler but it will do to bootstrap the process.
$ sudo apt-get install ocaml
2 INSTALLING & SET UP OPAM
OPAM is a useful package manager for OCaml code such as signposts. To download the initialize:
$ git clone git://github.com/OCamlPro/opam.git
$ cd opam && ./configure && make
$ sudo make install
$ opam init
$ eval 'opam config -env'
$ echo "$ which opam && eval 'opam config -env'" >> ~/.profile
3 SWITCH TO LATEST OCAML COMPILER
To see the compiler versions avaliable on OPAM use:
$ opam switch -list
To see the version of the OCaml compiler that is currently in use, use
$ ocaml -version
Then switch the OCaml 4.00.1 using
$ opam switch 4.00.1
$ eval 'opam config -env'
This process may take quite a while. You can now check the version of OCaml again, it it should show that the PATH is now pointing towards a new OCaml compiler version
4 REMOTE REPOSITORIES
To view the current remote respositories that OPAM is using, enter:
$ opam remote -list
This should give you the following output:
[curl] default http://opam.ocamlpro.com
To add the remote respositories required for signposts enter the following:
$ opam remote -kind git -add dev https://github.com/mirage/opam-repo-dev.git
$ opam remote -kind git -add mirage git://github.com/mirage/opam-repo.git
Checking the current remote repositories as before, now returns
[git] mirage git://github.com/mirage/opam-repo.git
[git] mirage-dev git://github.com/mirage/opam-repo-dev.git
[curl] default http://opam.ocamlpro.com
5 INSTALL LIBRARIES
To install the required packages from OPAM and the package manager use:
$ opam install lwt cstruct mirage mirage-net ocamlgraph uri rpc oasis ssl
$ sudo apt-get install libnfnetlink-dev libnl-3-dev libnl-cli-3-dev libnl-genl-3-dev libnl-nf-3-dev libnl-route-3-dev
Some the packages here are not the most upto date, to get the updates
$ git clone https://github.com/crotsos/mirage-platform.git
$ cd mirage-platform/
$ make all install
$ git clone https://github.com/crotsos/mirage-net.git
$ cd mirage-net/
$ make all install
$ git clone https://github.com/crotsos/ocaml-openflow.git
$ cd ocaml-openflow/
$ make all install
$ git clone https://github.com/crotsos/ocaml-dns.git
$ cd ocaml-dns/
$ make all install
$ git clone https://github.com/crotsos/ocaml-crypto-keys.git
$ cd ocaml-crypto-keys/
$ make all install
In the future, you update your packages using:
$ opam update
$ opam upgrade
6 GET SIGNPOST CODE
To download a copy of the Signpost Code using:
$ git clone https://github.com/crotos/signpostd
$ cd signpostd
$ make
7 VSWITCH
$ sudo wget https://www.dropbox.com/s/4n0hwgoycm3838g/openvswitch_mod.ko?dl=1 -O /lib/modules/`uname -r`/extra/openvswitch_mod.ko
$ sudo wget https://www.dropbox.com/s/f7ivv8upe0bfurf/brcompat_mod.ko?dl=1 -O /lib/modules/`uname -r`/extra/brcompat_mod.ko
$ sudo depmod -a
$ modprobe openvswitch_mod
$ sudo modprobe brocompat_mod
$ sudo ovs-vsctl add-br br0
$ sudo ovs-vsctl add-port br0 eth0
$ sudo ifconfig eth0 up
$ sudo ifconfig br0 up
$ sudo ovs-vsctl set-fail-mode br0 standalone
$ sudo ovs-vsctl set-controller br0 tcp:localhost
$ sudo ln -s /etc/init.d/openvswitch-switch /etc/rcS.d/S10openvswitch-switch
$ sudo chmod 777 /etc/network/interfaces
$ echo "pre-up ifconfig eth0 up" >> /etc/network/interfaces
8 KEY GENERATION
For the each client we wish to add to the signposts personal cloud we need to generate a private and associated key. To generate these key we will be using onpenssl. On each client we need to generate the private key and place it into the signposd/conf directory when we need to generate the public key from this and place on the server un signpostd/conf/authorized_keys
on the client
$ openssl genrsa -out conf/signpost.pem 2046
and on the server
$ openssl rsa -in conf/signpost.pem -pubout -out conf/authorized_keys/clientname.pub
9 IODINE
To install iodine from the ubuntu package manager
sudo apt-get install iodine
OpenWrt & Linksys WRT54GL Router - Meet & Greet
OpenWrt is a firmware for embedded devices used to router traffic. In this case we will be considering the use of OpenWRT in domestic routers such as the test hardware Linksys Wireless-G Broadband Router WRT54GL v1.1.
OpenWrt is Linux based so it included the Linux kernel as well as BusyBox. It has a package manager called opkg (similar to apt in ubuntu).
Before installing OpenWrt on a router, you must enable that the device is OpenWrt compatible, you can do this my ensuring the device is listed here
HARDWARE SPECIFICATIONS
Before exploring OpenWrt, We are going to take a closer look at the hardware available:
CPU: Broadcom BCM5352 @ 200 MHz
RAM: 16 MB
Flash Memory: 4 MB
QUICK CHECK - to ensure the hardware is what we believe it to be, we can check the prefix of the serial number using the information here
This hardware is fully supported by OpenWrt, but there have been issues with the limited amount of flash memory:
http://wiki.openwrt.org/toh/linksys/wrt54g#hardware
https://forum.openwrt.org/viewtopic.php?id=28223
The solution to this issues, has also been documented. This is to use OpenWrt 8.09 r14511 (code name "kamikaze") instead of the most up-to date version OpenWrt 10.03.1-rc6 (code name "backfire")
PICKING A VERSION
To start with we are going to install OpenWrt in Linksys Web GUI. There are many versions of OpenWrt available, so we need to identify to first version we will try:
INSTALLATION
Step 1: Download http://downloads.openwrt.org/kamikaze/8.09/brcm-2.4/openwrt-wrt54g-squashfs.bin to my Downloads directory
Step 2: Plug in router to mains and to computer via ethernet (use port 1 not internet port)
Step 3: Direct the browser to http://192.168.1.1 and log in
Step 4: Navigate to Administation > Firmware update, select openwrt-wrt54g-squashfs.bin and update
ALL IS LOOKING WELL :)
COMMUNICATION VIA WEB GUI
Direct the browser to http://192.168.1.1, log in and your presented with the web interface luci
COMMUNICATION VIA TELNET
The router should now be telnet(able) to 192.168.1.1. To test this:
$ telnet 192.168.1.1
This returns the recipe for KAMIKAZE :)
Now to ensure that tftp is available to prevent bricking, enter:
CONFIGURING
The network configuration is stored in /etc/config/network. The initial contents of this file for our set up is:
The content of the initial configuration file is
#### VLAN configuration
config switch eth0
option vlan0 "0 1 2 3 5*"
option vlan1 "4 5"
#### Loopback configuration
config interface loopback
option ifname "lo"
option proto static
option ipaddr 127.0.0.1
option netmask 255.0.0.0
#### LAN configuration
config interface lan
option type bridge
option ifname "eth0.0"
option proto static
option ipaddr 192.168.1.1
option netmask 255.255.255.0
#### WAN configuration
config interface wan
option ifname "eth0.1"
option proto dhcp
Once we have edited this file, to make the new configuration take after we need to :
$ /etc/init.d/network restart
SWITCH
The switch section of the above configuration file is responsible for making one peoice of hardware, appear as several independent interfaces. The part of the configuration file which specifies the switch characteristics is:
#### VLAN configuration
config switch eth0
option vlan0 "0 1 2 3 5*"
option vlan1 "4 5"
In the above configuration: The numbers 0-5 represent the port numbers, so VLAN0 includes ports 0 to 5* and VLAN1 includes ports 4 and 5. The * in 5*
indicates the PVID.
As shown in the above diagram, this switch separates the LAN ports and thWAN ports .
INTERFACES
The other statements in the configuration file describe the interfaces. The interfaces are logical networks, for the setting of IP address, routes and other magic.
The 3 interfaces that we have here are named loopback, lan and wan. The physical interfaces associated with these logical interfaces are lo, eth0.0 and eth0.1.
OpenWrt is Linux based so it included the Linux kernel as well as BusyBox. It has a package manager called opkg (similar to apt in ubuntu).
Before installing OpenWrt on a router, you must enable that the device is OpenWrt compatible, you can do this my ensuring the device is listed here
HARDWARE SPECIFICATIONS
Before exploring OpenWrt, We are going to take a closer look at the hardware available:
CPU: Broadcom BCM5352 @ 200 MHz
RAM: 16 MB
Flash Memory: 4 MB
QUICK CHECK - to ensure the hardware is what we believe it to be, we can check the prefix of the serial number using the information here
This hardware is fully supported by OpenWrt, but there have been issues with the limited amount of flash memory:
http://wiki.openwrt.org/toh/linksys/wrt54g#hardware
https://forum.openwrt.org/viewtopic.php?id=28223
The solution to this issues, has also been documented. This is to use OpenWrt 8.09 r14511 (code name "kamikaze") instead of the most up-to date version OpenWrt 10.03.1-rc6 (code name "backfire")
PICKING A VERSION
To start with we are going to install OpenWrt in Linksys Web GUI. There are many versions of OpenWrt available, so we need to identify to first version we will try:
- The OpenWrt version is Kamilaze, due to a bug in backfire and instability of attitude adjustment
- The recommended version is 8.09 within Kamilaze
- The CPU is broadcom so the prefix is bcrm
- For here, i can see the hardware supports both brcm-2.4 and brcm47xx
- The difference between brcm-2.4 and brcm47xx is explained here
- For ease, we will download a image file, this will end with .bin
- If both JFFS2 and SquashFS is available, use SpuashFS images
- Look into the version history to determine with version of 8.09 is best and what is different between kamikaze, backfire and attitude adjustment
INSTALLATION
Step 1: Download http://downloads.openwrt.org/kamikaze/8.09/brcm-2.4/openwrt-wrt54g-squashfs.bin to my Downloads directory
Step 2: Plug in router to mains and to computer via ethernet (use port 1 not internet port)
Step 3: Direct the browser to http://192.168.1.1 and log in
Step 4: Navigate to Administation > Firmware update, select openwrt-wrt54g-squashfs.bin and update
ALL IS LOOKING WELL :)
COMMUNICATION VIA WEB GUI
Direct the browser to http://192.168.1.1, log in and your presented with the web interface luci
COMMUNICATION VIA TELNET
The router should now be telnet(able) to 192.168.1.1. To test this:
$ telnet 192.168.1.1
This returns the recipe for KAMIKAZE :)
Now to ensure that tftp is available to prevent bricking, enter:
$ nvram set boot_wait=on $ nvram set boot_time=10 $ nvram commit && reboot
COMMUNICATION VIA SSH
CONFIGURING
The network configuration is stored in /etc/config/network. The initial contents of this file for our set up is:
The content of the initial configuration file is
#### VLAN configuration
config switch eth0
option vlan0 "0 1 2 3 5*"
option vlan1 "4 5"
#### Loopback configuration
config interface loopback
option ifname "lo"
option proto static
option ipaddr 127.0.0.1
option netmask 255.0.0.0
#### LAN configuration
config interface lan
option type bridge
option ifname "eth0.0"
option proto static
option ipaddr 192.168.1.1
option netmask 255.255.255.0
#### WAN configuration
config interface wan
option ifname "eth0.1"
option proto dhcp
Once we have edited this file, to make the new configuration take after we need to :
$ /etc/init.d/network restart
SWITCH
The switch section of the above configuration file is responsible for making one peoice of hardware, appear as several independent interfaces. The part of the configuration file which specifies the switch characteristics is:
#### VLAN configuration
config switch eth0
option vlan0 "0 1 2 3 5*"
option vlan1 "4 5"
In the above configuration: The numbers 0-5 represent the port numbers, so VLAN0 includes ports 0 to 5* and VLAN1 includes ports 4 and 5. The * in 5*
indicates the PVID.
As shown in the above diagram, this switch separates the LAN ports and thWAN ports .
INTERFACES
The other statements in the configuration file describe the interfaces. The interfaces are logical networks, for the setting of IP address, routes and other magic.
The 3 interfaces that we have here are named loopback, lan and wan. The physical interfaces associated with these logical interfaces are lo, eth0.0 and eth0.1.
Wednesday 7 November 2012
Lots of things to be getting on with ...
Friends often wonder what I'm working (and so do I sometimes) so here's my to-do list for the rest of this week:
DOCUMENTING OCAML
DOCUMENTING OCAML
- Review some examples of ocaml code and hows its documented
- Investigate if ocamldoc is worth implementing
- using ocaml-dns as an example of documentation
- put together a blog post of methods of documenting OCaml code and decide a style of documentation that you be useful in Signposts
- Next week, work through the core signposts core, adding documentation
- working through the process of installing signposts from a scratch, currently do as far as key generation
- Generate the keys required for signpost and potential write a script to help automate the process
- Run Signposts on my main machine, test client and server implementation
- Next week, follow documentation to set up signposts on an Eee PC
- Produce a clear set on instruction on how to install, set up and run signposts
UPnP TACTIC FOR SIGNPOSTS
- Research the UPnP interface provided by routers
- Review the other signposts tactics to generate an outline of the API for implementing tactics
- Next week, Write a tactic in OCaml for signposts that makes use of UPnP, use the method of documenting OCaml highlights earlier
- finish reading and taking notes on "Usage Patterns in an Urban Wi-Fi network"
- Produce a draft outline of the framework for analyzing urban Wi-Fi
- Next week, reading though some other papers in the area and add detail to framework
- once permission for the network is granted, set up the router using the provided interface
- install and set-up OpenWRT on the router
- Investigate best firmware for ftp, ssh and vpn on the router
- produce results and share, before 15th Nov
- consider the overlap of signposts & paws
- find use cases that highlight the potential for inter connection
- produce some slides/blog post on findings and ideas, before 15th Nov
Tuesday 2 October 2012
DNSSEC - A basic Intro
This article aims to not just explain how DNSSEC work but why the protocols are designed the way that they are. I will begin with a simple mental model of public-key cryptography, then I highlight issues with scheme described so far and add complexity to fix issues as I highlight them. I finish the article with a simplified model of DNSSEC and a range of problems/question that I hope to explore in the next article.
DNSSEC is a collection of IETF specifications for securing DNS data. It is a set of extensions to DNS which provide to DNS resolvers with:
DNSSEC aims to adds security to DNS, while maintaining backwards compatibility.
RFC 3833 <- Threat Analysis of DNS and why DNSSEC was required
DNSSEC protects DNS resolvers from forged DNS data, by digitally signing all DNSSEC replies. The DNS resolver is then able to check if the information it receives is identical to that information on the authoritative DNS server.
DNSSEC can also protect information such as cryptographic certificates stored in CERT records in DNS.
RFC 4398 <- Storing certificates in DNS
RFC 4367 <- Whats in a Name (looks very interesting)
Negative article on DNSSEC
DNSSEC-bis is the name of the DNSSEC specification and the related RFCs are
4033 introduction & requirements, 4034 resource records for DNSSEC and 4035 on protocol modifications.
Public-key cryptography is the use of two keys, one public and one private to encrypt and decrypt a message. A message before encryption is referred to as plain text and a message after encryption is referred to as cipher text.
Public-key cryptography can be used to provide confidentiality or authentication.
To provide confidentiality, the public key is used for encryption so anyone can write a message but only the intended recipient has the private key which is used for decryption, so the only person who can decrypt the message is intended recipient.
To provide origin authentication, the private key is used for encryption so the only person who can send an encrypted message is the intended source. Then the public key is used for decryption so anyone can read the message, provided it was written by the intended source.
DNSSEC works by digitally signing the results of DNS look-ups using public-key cryptography. It uses public-key cryptography to provide authentication not confidentiality. This means that private key is used for encryption and the public key is used for decryption. So when a DNS resolver makes a DNSSEC query, then the response will be signed by the correct information source and then the DNS resolver will use the public key to decrypt the response, knowing the response is reliable as it was definitely send by the intended source.
So that's the problem solved, the DNS responses will be encrypted by correct source and then all the DNS resolvers will decryption the responses using the public key. This ensures that the response was sent by the correct source so it provides source authentication as well an data integrity as it was signed using a private key.
Except that this model completely ignores the details of how DNS actually generates the replies to DNS query and what exactly is the "intended source" in this context.
So now I want to explain why the basic public-key cryptography scheme that I outline is not sufficient to provide authentication.
The cryptography scheme aimed to provide authentication between the DNS resolver and the source of the DNS response, but it didn’t consider the details of how this DNS response was generated.
The actually answer is that the "intended source" in DNS is an Authoritative Name Server, There are many authoritative name server forming a distributed database, each authoritative name server responses to a specific subset of the DNS queries.
The DNS resolver needs to locate the relevant authoritative name server, this is achieved through a chain of domain name servers, either recursively, iteratively or a hybrid of them both.
To generate the response to a DNS query for the DNS resolver, there are a series of request-replies. So for DNSSEC to meet its primary goal of providing authenticated responses, there needs to be authentication throughout the chain between the DNS resolver and the intended source, which is the authoritative name server. When a DNSSEC enabled DNS resolver gets a response then it can use the public key to check that it was encrypted by the authoritative name server.
So now we have fixed the problem ... Actually not
This solution would require that all DNS resolvers change because all DNS resolvers need to decrypt the DNSSEC response using the public key to get the the normal DNS response. Given that DNSSEC nees to be backwards compatible with DNS, this means we need to alter the above scheme.
So instead of encrypting and decrypting the whole DNS response, why don't we add some text to the DNS response as another resource record and then encrypt and decrypt that text instead. This means that DNS resolvers that are not DNSSEC compatible can simply ignore the extra resource records. This extra bit of information therefore acts as a digital signature. This is actually used in DNSSEC and the new resource record is called RRSIG.
Now we have DNS record with an extra resource record called RRSIG, the authoritative name server has the private key and the DNS resolver use the public key to check that the DNS information received was from an authoritative name server. But where do the DNS resolvers get the public key from ?
A simple idea would be to have a single server with all public keys on, then all DNSSEC enabled resolvers could just request the public key from the central server. This is fundamentally flawed as its a centralized system which would completely defeat the purpose of DNS, which is to act as a distributed system.
Therefore we need a distributed system of our own to get the public keys to the DNSSEC enabled resolvers, why not use a system like DNS or even DNSSEC ?
DNSSEC gets the public key, from the authoritative name server to the DNSSEC enabled resolver by introducing another new resource record called DNSKEY. This has the public key in it so that the DNS resolver can use it to verify the RRSIG.
Done ? Nope of course not ...
So its easy to assume that we finished here but as we developed the scheme we have actually introduced new issues, for example we have lost any data integrity. Its easy for a 3rd party to intersect a DNS query, change the data in the resources and set it forward to its destination. The DNSSEC enabled resolver would be nun the wiser, as the response to the DNSSEC query is still correctly signed and will have the correct RRSIG.
We can fix they by making the plain text in the RRSIG, a hash of the original message. This means that if the messaged is changed in transit then the DNS resolver will detect it because after decrypting the RRSIG text and hashing the other resource records, the results will no longer be the same because the hash generated from the DNSSEC response will be different to the RRSIG one as the text has been changed.
Lets review where we have got to so far...
DNSSEC uses public-key cryptography to provide authentication and data integrity between authoritative name servers and DNS resolvers. It does this using two new resource records, RRSIG and DNSKEY.
The authoritative name server hashes the RRs and then uses a private key to encode this hash, the result is placed in the RRSIG record and the public key is placed in the DNSKEY record. Then the DNS response moves though a chain of domain name servers to the DNS resolver. If the DNS resolver supports DNSSEC then it will use the public key in the DNSKEY record to decrypt the RRSIG, this will give a hash of the other RRs, the resolver can then hash the other RRs and compare the results.
So what is this public-key cryptography algorithm that DNSSEC uses and whats going to happen if its broken in the future ?
DNSSEC doesn’t use a specific algorithm. Both DNSKEY and RRSIG have a 8 bit number, which corresponds to the algorithm used, details of with values correspond to which algorithms is available here
This scheme has a big problem though...
Consider the following situation, I am an evil 3rd party would wants to return the incorrect DNS respond to a (DNSSEC enable) DNS resolver :
If we assume for a moment that the DNSKEY is authenticated, then a (DNSSEC enable) DNS resolver will be able to detect if the other resource records have been changed. As the decrypted RRSIG will not be a hash of the other resource records. If a evil 3rd party also when tries to also change the RRSIG value, so that the decrypted RRSIG is a hash of the changes made, then they can do this only if they know the private key. But they do not know the private key used and they cannot replace the public/private key pair as we have assumed that the DNSKEY is authenticated
So how can we ensure that the DNSKEY is from the authoritative name server ?
DNSSEC works around this is by signing the DNS replies at each step in the chain of response and reply. So when a domain name server queries another domain name server, not only is the expected result returned but also a flag indicating if that zone used DNSSEC and if it does it includes a hash of the DNSKEY record stored in a record called DS.
So far we have introduced 3 new resource records to DNS:
This hash is then used when the name server queries the zone, as the DNSKEY returned hashed should match the DS RR from the previous query. If the response doesn’t include a DNSSEC RR or the DNSSEC hashed is not the same as the previously returned DS RR, then the integrity of the data has been comprised.
Example
This diagram attempt to demonstrate how the chain of DS and DNSKEY allows use to assume that the DNSKEY has not been tampered with. In this example, a DNS resolver queries the root DNS name server and its returned with the normal DNS response as well as a DS resource record. This DS RR is a hash of the DNSKEY that will be returned then the next DNS request is made. The DNS resolver then queries a TLD name server, this returns the normal DNS response as well as a DNSKEY and another DS. Now the DNS resolver can check that the response was from the TLD name server by checking that the DS that is recieved prevously from the root DNS name server is a hash of the DNSKEY. The DNS resolver can now contiune as it has the DS, which is a hash of the DNSKEY that should be returned by the next name server
How does the DNS resolver know that it can trust the result from the root DNS server ?
In the above example we cant guarantee that response from the root DNS server is actually from the root DNS server, this is why we introduce a trust anchor
But this appears just to move the problem, how can we trust the trust anchor ?
We get the DS resource record using a different method where we don't need to authenticate the origin as we know that the message can't have been intercepted (i.e. via the operating system)
As you can see we have created a chain of trust using a series of corresponding DS records and DNSKEYs, this means that when we reach the authoriative DNS name server, we know that we can trust the DNSKEY and that is will not have been tampered with.
There is a fun website here, which provides information on the chain of trust.
Thing to consider in the next article:
DNSSEC is a collection of IETF specifications for securing DNS data. It is a set of extensions to DNS which provide to DNS resolvers with:
- origin authentication of DNS data
- authenticated denial of existence
- data integrity
- availability
- confidentiality, DNSSEC results are not encrypted
- protection from user mistakes/assumptions
DNSSEC aims to adds security to DNS, while maintaining backwards compatibility.
RFC 3833 <- Threat Analysis of DNS and why DNSSEC was required
DNSSEC protects DNS resolvers from forged DNS data, by digitally signing all DNSSEC replies. The DNS resolver is then able to check if the information it receives is identical to that information on the authoritative DNS server.
DNSSEC can also protect information such as cryptographic certificates stored in CERT records in DNS.
RFC 4398 <- Storing certificates in DNS
RFC 4367 <- Whats in a Name (looks very interesting)
Negative article on DNSSEC
DNSSEC-bis is the name of the DNSSEC specification and the related RFCs are
4033 introduction & requirements, 4034 resource records for DNSSEC and 4035 on protocol modifications.
Public-key cryptography is the use of two keys, one public and one private to encrypt and decrypt a message. A message before encryption is referred to as plain text and a message after encryption is referred to as cipher text.
Public-key cryptography can be used to provide confidentiality or authentication.
To provide confidentiality, the public key is used for encryption so anyone can write a message but only the intended recipient has the private key which is used for decryption, so the only person who can decrypt the message is intended recipient.
To provide origin authentication, the private key is used for encryption so the only person who can send an encrypted message is the intended source. Then the public key is used for decryption so anyone can read the message, provided it was written by the intended source.
 |
| Figure 1 - How public key cryptography can be used to provide authentication or confidentiality |
DNSSEC works by digitally signing the results of DNS look-ups using public-key cryptography. It uses public-key cryptography to provide authentication not confidentiality. This means that private key is used for encryption and the public key is used for decryption. So when a DNS resolver makes a DNSSEC query, then the response will be signed by the correct information source and then the DNS resolver will use the public key to decrypt the response, knowing the response is reliable as it was definitely send by the intended source.
So that's the problem solved, the DNS responses will be encrypted by correct source and then all the DNS resolvers will decryption the responses using the public key. This ensures that the response was sent by the correct source so it provides source authentication as well an data integrity as it was signed using a private key.
Except that this model completely ignores the details of how DNS actually generates the replies to DNS query and what exactly is the "intended source" in this context.
So now I want to explain why the basic public-key cryptography scheme that I outline is not sufficient to provide authentication.
The cryptography scheme aimed to provide authentication between the DNS resolver and the source of the DNS response, but it didn’t consider the details of how this DNS response was generated.
The actually answer is that the "intended source" in DNS is an Authoritative Name Server, There are many authoritative name server forming a distributed database, each authoritative name server responses to a specific subset of the DNS queries.
The DNS resolver needs to locate the relevant authoritative name server, this is achieved through a chain of domain name servers, either recursively, iteratively or a hybrid of them both.
To generate the response to a DNS query for the DNS resolver, there are a series of request-replies. So for DNSSEC to meet its primary goal of providing authenticated responses, there needs to be authentication throughout the chain between the DNS resolver and the intended source, which is the authoritative name server. When a DNSSEC enabled DNS resolver gets a response then it can use the public key to check that it was encrypted by the authoritative name server.
So now we have fixed the problem ... Actually not
This solution would require that all DNS resolvers change because all DNS resolvers need to decrypt the DNSSEC response using the public key to get the the normal DNS response. Given that DNSSEC nees to be backwards compatible with DNS, this means we need to alter the above scheme.
So instead of encrypting and decrypting the whole DNS response, why don't we add some text to the DNS response as another resource record and then encrypt and decrypt that text instead. This means that DNS resolvers that are not DNSSEC compatible can simply ignore the extra resource records. This extra bit of information therefore acts as a digital signature. This is actually used in DNSSEC and the new resource record is called RRSIG.
Now we have DNS record with an extra resource record called RRSIG, the authoritative name server has the private key and the DNS resolver use the public key to check that the DNS information received was from an authoritative name server. But where do the DNS resolvers get the public key from ?
A simple idea would be to have a single server with all public keys on, then all DNSSEC enabled resolvers could just request the public key from the central server. This is fundamentally flawed as its a centralized system which would completely defeat the purpose of DNS, which is to act as a distributed system.
Therefore we need a distributed system of our own to get the public keys to the DNSSEC enabled resolvers, why not use a system like DNS or even DNSSEC ?
DNSSEC gets the public key, from the authoritative name server to the DNSSEC enabled resolver by introducing another new resource record called DNSKEY. This has the public key in it so that the DNS resolver can use it to verify the RRSIG.
Done ? Nope of course not ...
So its easy to assume that we finished here but as we developed the scheme we have actually introduced new issues, for example we have lost any data integrity. Its easy for a 3rd party to intersect a DNS query, change the data in the resources and set it forward to its destination. The DNSSEC enabled resolver would be nun the wiser, as the response to the DNSSEC query is still correctly signed and will have the correct RRSIG.
We can fix they by making the plain text in the RRSIG, a hash of the original message. This means that if the messaged is changed in transit then the DNS resolver will detect it because after decrypting the RRSIG text and hashing the other resource records, the results will no longer be the same because the hash generated from the DNSSEC response will be different to the RRSIG one as the text has been changed.
Lets review where we have got to so far...
DNSSEC uses public-key cryptography to provide authentication and data integrity between authoritative name servers and DNS resolvers. It does this using two new resource records, RRSIG and DNSKEY.
The authoritative name server hashes the RRs and then uses a private key to encode this hash, the result is placed in the RRSIG record and the public key is placed in the DNSKEY record. Then the DNS response moves though a chain of domain name servers to the DNS resolver. If the DNS resolver supports DNSSEC then it will use the public key in the DNSKEY record to decrypt the RRSIG, this will give a hash of the other RRs, the resolver can then hash the other RRs and compare the results.
So what is this public-key cryptography algorithm that DNSSEC uses and whats going to happen if its broken in the future ?
DNSSEC doesn’t use a specific algorithm. Both DNSKEY and RRSIG have a 8 bit number, which corresponds to the algorithm used, details of with values correspond to which algorithms is available here
This scheme has a big problem though...
Consider the following situation, I am an evil 3rd party would wants to return the incorrect DNS respond to a (DNSSEC enable) DNS resolver :
- The (DNSSEC enable) DNS resolver places a DNS request for www.google.com
- I spot this and I want to send a reply to this DNS request that will not be from the authoritative name server, managed by Google
- So I'm going to make my own response and make it appear as its has comes from the authoritative name server
- I create RR with the incorrect information and I then come up with my own public and private key pair, using the private key to encrypt the hash in the RRSIG and then place the public key in the DNSKEY record
- The DNS resolver then decrypts the RRSIG record using the key in the DNSKEY and checks that its a hash of the other RR records
- The DNS resolver believes that the response is from the authoritative name server at google
If we assume for a moment that the DNSKEY is authenticated, then a (DNSSEC enable) DNS resolver will be able to detect if the other resource records have been changed. As the decrypted RRSIG will not be a hash of the other resource records. If a evil 3rd party also when tries to also change the RRSIG value, so that the decrypted RRSIG is a hash of the changes made, then they can do this only if they know the private key. But they do not know the private key used and they cannot replace the public/private key pair as we have assumed that the DNSKEY is authenticated
So how can we ensure that the DNSKEY is from the authoritative name server ?
DNSSEC works around this is by signing the DNS replies at each step in the chain of response and reply. So when a domain name server queries another domain name server, not only is the expected result returned but also a flag indicating if that zone used DNSSEC and if it does it includes a hash of the DNSKEY record stored in a record called DS.
So far we have introduced 3 new resource records to DNS:
- RRSIG - a hash of the other RRs that is encrypt using the private key by the authoritative name server
- DNSKEY - the public key associated with the private key used to encrypt the RRSIG, a DNS resolver then using this public key to decypt the RRSIG and check that the plaintext is a hash of the other RRs
- DS - a hash of the DNSKEY used in the zone that the DNS resolver is about to query so that the resolver can check that the response from the next name server query has not had the DNSKEY altered because it can check the DNSKEY received with DS received from the last name server.
This hash is then used when the name server queries the zone, as the DNSKEY returned hashed should match the DS RR from the previous query. If the response doesn’t include a DNSSEC RR or the DNSSEC hashed is not the same as the previously returned DS RR, then the integrity of the data has been comprised.
Example
This diagram attempt to demonstrate how the chain of DS and DNSKEY allows use to assume that the DNSKEY has not been tampered with. In this example, a DNS resolver queries the root DNS name server and its returned with the normal DNS response as well as a DS resource record. This DS RR is a hash of the DNSKEY that will be returned then the next DNS request is made. The DNS resolver then queries a TLD name server, this returns the normal DNS response as well as a DNSKEY and another DS. Now the DNS resolver can check that the response was from the TLD name server by checking that the DS that is recieved prevously from the root DNS name server is a hash of the DNSKEY. The DNS resolver can now contiune as it has the DS, which is a hash of the DNSKEY that should be returned by the next name server
 |
| Figure 2 - DNSSEC chain of trust created using corresponding DS and DNSKEY RRs, this assumes that the response from the root DNS can be trusted |
How does the DNS resolver know that it can trust the result from the root DNS server ?
In the above example we cant guarantee that response from the root DNS server is actually from the root DNS server, this is why we introduce a trust anchor
 |
| Figure 3 - chain of trust between a trush anchor and the authoriatative name server using a series of DS and RRSIG resource records |
We get the DS resource record using a different method where we don't need to authenticate the origin as we know that the message can't have been intercepted (i.e. via the operating system)
As you can see we have created a chain of trust using a series of corresponding DS records and DNSKEYs, this means that when we reach the authoriative DNS name server, we know that we can trust the DNSKEY and that is will not have been tampered with.
There is a fun website here, which provides information on the chain of trust.
Thing to consider in the next article:
- What's the difference between a validating and non-validating stud resolver, which is better ?
- DNSSEC also introduces NSEC and NSEC3 resource records, what are these used for ?
- DNSSEC requires use of the "DO" flag in EDNS, why don't the DNS resolvers just ignore the RRSIG. DNSKEY and DS resource records ?
- What happens if the authoriative name server do not support DNSSEC, how does the resolver know the difference between DNS response from an authoriative name server that don't use DNSSEC and a attach where the evil 3rd party tries to pretent the the authoriative name server dose not use DNSSEC
- How exactly goes the trust anchor get the DS for DNS root, what happens if the root DNS needs to update the DNSKEY and therefore the DS record in the trust anchor, if for example the cyptographic algorithm is broken
- How does a authoriative name server prove that an DNS entery don't exist ?
- In July 2010, the DNSSEC was added to the root DNS, could you use DNSSEC before the root DNS support DNSEC if all other name server and the resolver supported DNSSEC ? DNS Lookaside validation ?
- What happens when a name server between the root and authoriate name server doesnt support DNSSEC but the other name server involved do ?
- Why are there additional timestamps in the RRSIG to limit the time that the signiture is valid ?
- Why are these timestamps absolute not relative like TTLs ?
- How are new keys implemented and what about the caching?
- Whats zome emuation and how can it be dealt with ?
Thursday 23 August 2012
Notes on "A Close Examination of Performance and Power Characteristics of 4G LTE Networks"! - Pt1
These are my notes on "A Close Examination of Performance and Power
Characteristics of 4G LTE Networks" which can be found here
4G is as the name suggests, the forth generation of mobile communication standards. There are currently two competing technologies, Mobile WiMAX and LTE. Long term evolution (LTE) is the technology that this paper considers.
4G Test Android Application
The team designed an android application called 4GTest. It is available in the Play store here. The application measures the network performance including latency, download speed and upload speed.
The application description claim that the results produced by this application are more accurate that any other test using single-threaded TCP tests.
I tested the application on an old HTC Magic but I couldn't get the application to recognize my internet connection. The application is not compatible with the Nexus 7.
Overall Results/ Conclusions
The abstract states that "LTE generally has significantly higher downlink and uplink throughput than 3G and even WiFi, with a median value of 13Mbps and 6Mbps, respectively". I am not sure why the median value has been quoted here instead of the mean value (if you know why, please comment below).
The data set for the study was 20 smart phone users over 5 months, this seems like a fairly limited sample size.
The power model of the LTE network, had a less than 6% error rate. It was found that LTE was less power efficient than WiFi and 3G. Testing of android applications highlighted that the device power is now becoming a bottleneck, instead of network performance, compared to 3G.
Long term evolution
LTE aims to enhance the Universal Terrestrial Radio Access Network, more commonly known as 3G. LTE is commercially avaliable but is not yet wide spread. The targeted user throughput is 100Mbps for downlink and 50Mbps for uplink. These targets are distinctly different to median values of 13Mbps and 6Mbps, respectively previously quoted.
User-plane latency is defined as the one-way transit time between the availability of a packet at the IP layer (the network layer) at the source and the availability of this packet at the IP layer (the network layer) at the destination. This definition means that user-plane latency includes the delay introduced by associated protocols. Since the delay is measured from network layer to network layer, we do not need to consider the delay introduced by the application layer or transport layer.
LTE can be compared to other networks such as Wi-FI and 3G by comparing network statistic such as bit rate, latency, user equipment (UE) power saving etc..
LTE uses Orthogonal Frequency Division Multiplex (OFDM) technology.
OFDM is based on FDM, but FDM wastes bandwidth as you need to leave bandwidth free between different carriers to stop the signals from interfering with each other. OFDM allows the carriers to be more closely spaced together so less bandwidth is wasted.
But OFDM is less power efficient as its more complex and requires linear amplification. To save power, LTE up ink uses a special implementation of OFDM called SC-FDMC.
Discontinuous reception (DRX) is also employed to reduce UE power consumption by some existing mobile technologies. LTE supports DRX. DRX is configured on a per UE basis and allows tradeoff to be made between power saving, delay and signalling overhead.
This study differs from previous ones since it doesn’t use either total on time or simplified LTE. Furthermore it uses real user traces instead of synthetic packets. UMICH refers to the real user data set of 20 smartphone users over 5 months, this data consists of user traces from Wi-Fi and 3G networks but not from LTE networks. So instead Wi-Fi traces are feed into the LTE model simulation framework, Wi-Fi traces were chosen over 3G as the RRT of Wi-Fi is more similar to that of LTE than 3G.
The study shows that LTE is less power efficient than WiFi and 3G for small data transfers but for bulk data transfers LTE is more power efficient than 3G but not Wi-Fi. Because LTE is more efficient than 3G for bulk data transfer then its important to make us of application tools such as Application Resource Optimizer (ARO) (MobiSys11 demo here) in LTE.
LTE is less power efficient than WiFi and 3G, even when DRX was used.
In LTE, the tail timer is the key parameter in determining the trade off between UE energy usage, performance and signalling overhead.
The study identified performance bottlenecks for android applications caused by device processing power, this is detected by monitoring CPU usage of application on the android devices.
Background on Radio Resource Control (RRC) and Discontinuous Reception (DRX) in LTE
Radio Resource Control (RRC) is a signalling protocol between user equipment (UE) and the 3G network (or 4G in this case). Long term evolution (LTE) has two RRC states: connected and idle. The transition from RRC-connected to RRC-idle is made when no data have been received/sent in a time peroid. The transition from RRC-idle to RRC-connection is made when some data is received/sent
Discontinuous Reception (DRX) is a used in mobile communication to conserve power. The UE and the network (in this case LTE) decide on phases when data transfer occurs, outside of these phases the network reciever on the mobile device is turned off thus consuming less energy.
For example. in 802.11 wireless networks, polling is used to control DRX. The mobile device is placed into standby for a set time interval and then a message is sent by the access point to indicate if there is any waiting data, if not then it is placed in standby again.
When LTE is RRC-connected state, then the UE can be in continuous reception, short DRX or long DRX. When LTE is RRC-idle, then the UE is only in DRX mode. Long DRX and short DRX are cycles of the receiver being on and off. The receiver is off for longer in long DRX, which increases delay but reduces emergy consuption. The receiver is on for longer in short DRX, which increases energy consuption but reduces deley. The parameters which dictate the length of time before various transitions, controls the trade-off between battery saving and latency.
Network Measurement
The android application designed to collect test data was called 4GTest and allows the user to switch between 3G, WiFI and LTE. The application made use of M-lab support. Measurement Lab (M-lab) is an distributed server platform for the deployment of internet measurement tools. The server-side tools are open source and the API's allow researchers to develop client tools such as 4GTest.
When considering RRT for LTE, its important to consider the latency of the wired parts of the path from client to server because the latency of LTE is lower than that of 3G so errors (caused by wired parts) become more notable. To minimize the path from client to server, which is not LTE, the nearest server to the client is always used.
If GPS was not available then IP address was used to locate the client. This translation from IP address to GPS is a rough estimate. The service used for this translation was MaxMind.
To measure RRT and latency variation (difference in latency between connections not packets so its not called Jitter), the application repeatedly established a new TCP connection with the server and measures the delay between SYN and SYN-ACK packet. The median RTT measurements and the variation are reported to central server
To measure peek channel capacity, 4GTest sets up multi-threaded TCP measurements using the 3 nearest servers in M-lab. The throughput test lasts for 20 seconds, the first 5 seconds were ignored due to TCP slow start then the other 15 sec are split into 15 1 sec bins. The average throughput for each bin is calculated and the median of all bins is the measured throughput.
Interestingly, the median is used here instead of average as median reduces the impact of abnormal bins.
To collect packet traces on the android phone, tcpdump was cross-compiled.
To capture CPU usage history, a C program was written to read /proc/stat in the android system.
To understand the impact of packet size on one-way delay, delay was measured for a range of packet sizes and for each packet size, 100 samples were measured.
Power measurements were made using a Monsoon power monitor as power input for the android phone. Phone screen was turned off where possible.
The data set used for analysis is called UMICH and includes full packet traces in TCPdump format including both headers and payloads
The network model simulator takes the binary packet trace files in libpcap format and percessioning is required to collect accurate data.
Application performance
The sample applications tested where some of the most popular apps. For the browser a simple website and a content-rich website were both tested. The approach taken involved launch the applications and then monitoring upstream, downstream data and CPU usage.
LTE Network characterization
These results are from the public deployment of 4GTest. In the US, the coverage of LTE, WiMAX and WiFi were found by 4GTest to be similar.
The public deployment of 4GTest found that the downlink and uplink throughput were both much higher for LTE than for Wi-FI, WiMAX and 3G technology. High variation in LTE throughput was observed.
For RRT and Jitter, LTE was found to have similar values to WiFi and better values than 3G and WiMAX
One-way delay and impact of packet size
On Wifi, packet size didn’t seem to influence one way delay, in either direction. On LTE, the uplink delay increases as packet size increases.
Modility
The requirements for LTE highlight that the network should be optimized for UE at low mobile speed (0 to 15 km/h) whilst still supporting high mobile speed (15 to 120 km/h). It was observed that in the face of speed changes, RTT remains stable and no significant change in throughput was observed.
Time of Day
Time of day was not found to have significant impact on the observed results .
Characteristics of 4G LTE Networks" which can be found here
4G is as the name suggests, the forth generation of mobile communication standards. There are currently two competing technologies, Mobile WiMAX and LTE. Long term evolution (LTE) is the technology that this paper considers.
4G Test Android Application
The team designed an android application called 4GTest. It is available in the Play store here. The application measures the network performance including latency, download speed and upload speed.
The application description claim that the results produced by this application are more accurate that any other test using single-threaded TCP tests.
I tested the application on an old HTC Magic but I couldn't get the application to recognize my internet connection. The application is not compatible with the Nexus 7.
Overall Results/ Conclusions
The abstract states that "LTE generally has significantly higher downlink and uplink throughput than 3G and even WiFi, with a median value of 13Mbps and 6Mbps, respectively". I am not sure why the median value has been quoted here instead of the mean value (if you know why, please comment below).The data set for the study was 20 smart phone users over 5 months, this seems like a fairly limited sample size.
The power model of the LTE network, had a less than 6% error rate. It was found that LTE was less power efficient than WiFi and 3G. Testing of android applications highlighted that the device power is now becoming a bottleneck, instead of network performance, compared to 3G.
Long term evolution
LTE aims to enhance the Universal Terrestrial Radio Access Network, more commonly known as 3G. LTE is commercially avaliable but is not yet wide spread. The targeted user throughput is 100Mbps for downlink and 50Mbps for uplink. These targets are distinctly different to median values of 13Mbps and 6Mbps, respectively previously quoted.
User-plane latency is defined as the one-way transit time between the availability of a packet at the IP layer (the network layer) at the source and the availability of this packet at the IP layer (the network layer) at the destination. This definition means that user-plane latency includes the delay introduced by associated protocols. Since the delay is measured from network layer to network layer, we do not need to consider the delay introduced by the application layer or transport layer.
LTE can be compared to other networks such as Wi-FI and 3G by comparing network statistic such as bit rate, latency, user equipment (UE) power saving etc..
LTE uses Orthogonal Frequency Division Multiplex (OFDM) technology.
OFDM is based on FDM, but FDM wastes bandwidth as you need to leave bandwidth free between different carriers to stop the signals from interfering with each other. OFDM allows the carriers to be more closely spaced together so less bandwidth is wasted.
But OFDM is less power efficient as its more complex and requires linear amplification. To save power, LTE up ink uses a special implementation of OFDM called SC-FDMC.
Discontinuous reception (DRX) is also employed to reduce UE power consumption by some existing mobile technologies. LTE supports DRX. DRX is configured on a per UE basis and allows tradeoff to be made between power saving, delay and signalling overhead.
This study differs from previous ones since it doesn’t use either total on time or simplified LTE. Furthermore it uses real user traces instead of synthetic packets. UMICH refers to the real user data set of 20 smartphone users over 5 months, this data consists of user traces from Wi-Fi and 3G networks but not from LTE networks. So instead Wi-Fi traces are feed into the LTE model simulation framework, Wi-Fi traces were chosen over 3G as the RRT of Wi-Fi is more similar to that of LTE than 3G.
The study shows that LTE is less power efficient than WiFi and 3G for small data transfers but for bulk data transfers LTE is more power efficient than 3G but not Wi-Fi. Because LTE is more efficient than 3G for bulk data transfer then its important to make us of application tools such as Application Resource Optimizer (ARO) (MobiSys11 demo here) in LTE.
LTE is less power efficient than WiFi and 3G, even when DRX was used.
In LTE, the tail timer is the key parameter in determining the trade off between UE energy usage, performance and signalling overhead.
The study identified performance bottlenecks for android applications caused by device processing power, this is detected by monitoring CPU usage of application on the android devices.
Background on Radio Resource Control (RRC) and Discontinuous Reception (DRX) in LTE
Radio Resource Control (RRC) is a signalling protocol between user equipment (UE) and the 3G network (or 4G in this case). Long term evolution (LTE) has two RRC states: connected and idle. The transition from RRC-connected to RRC-idle is made when no data have been received/sent in a time peroid. The transition from RRC-idle to RRC-connection is made when some data is received/sent
Discontinuous Reception (DRX) is a used in mobile communication to conserve power. The UE and the network (in this case LTE) decide on phases when data transfer occurs, outside of these phases the network reciever on the mobile device is turned off thus consuming less energy.
For example. in 802.11 wireless networks, polling is used to control DRX. The mobile device is placed into standby for a set time interval and then a message is sent by the access point to indicate if there is any waiting data, if not then it is placed in standby again.
When LTE is RRC-connected state, then the UE can be in continuous reception, short DRX or long DRX. When LTE is RRC-idle, then the UE is only in DRX mode. Long DRX and short DRX are cycles of the receiver being on and off. The receiver is off for longer in long DRX, which increases delay but reduces emergy consuption. The receiver is on for longer in short DRX, which increases energy consuption but reduces deley. The parameters which dictate the length of time before various transitions, controls the trade-off between battery saving and latency.
Network Measurement
The android application designed to collect test data was called 4GTest and allows the user to switch between 3G, WiFI and LTE. The application made use of M-lab support. Measurement Lab (M-lab) is an distributed server platform for the deployment of internet measurement tools. The server-side tools are open source and the API's allow researchers to develop client tools such as 4GTest.
When considering RRT for LTE, its important to consider the latency of the wired parts of the path from client to server because the latency of LTE is lower than that of 3G so errors (caused by wired parts) become more notable. To minimize the path from client to server, which is not LTE, the nearest server to the client is always used.
If GPS was not available then IP address was used to locate the client. This translation from IP address to GPS is a rough estimate. The service used for this translation was MaxMind.
To measure RRT and latency variation (difference in latency between connections not packets so its not called Jitter), the application repeatedly established a new TCP connection with the server and measures the delay between SYN and SYN-ACK packet. The median RTT measurements and the variation are reported to central server
To measure peek channel capacity, 4GTest sets up multi-threaded TCP measurements using the 3 nearest servers in M-lab. The throughput test lasts for 20 seconds, the first 5 seconds were ignored due to TCP slow start then the other 15 sec are split into 15 1 sec bins. The average throughput for each bin is calculated and the median of all bins is the measured throughput.
Interestingly, the median is used here instead of average as median reduces the impact of abnormal bins.
To collect packet traces on the android phone, tcpdump was cross-compiled.
To capture CPU usage history, a C program was written to read /proc/stat in the android system.
To understand the impact of packet size on one-way delay, delay was measured for a range of packet sizes and for each packet size, 100 samples were measured.
Power measurements were made using a Monsoon power monitor as power input for the android phone. Phone screen was turned off where possible.
The data set used for analysis is called UMICH and includes full packet traces in TCPdump format including both headers and payloads
The network model simulator takes the binary packet trace files in libpcap format and percessioning is required to collect accurate data.
Application performance
The sample applications tested where some of the most popular apps. For the browser a simple website and a content-rich website were both tested. The approach taken involved launch the applications and then monitoring upstream, downstream data and CPU usage.
LTE Network characterization
These results are from the public deployment of 4GTest. In the US, the coverage of LTE, WiMAX and WiFi were found by 4GTest to be similar.
The public deployment of 4GTest found that the downlink and uplink throughput were both much higher for LTE than for Wi-FI, WiMAX and 3G technology. High variation in LTE throughput was observed.
For RRT and Jitter, LTE was found to have similar values to WiFi and better values than 3G and WiMAX
One-way delay and impact of packet size
On Wifi, packet size didn’t seem to influence one way delay, in either direction. On LTE, the uplink delay increases as packet size increases.
Modility
The requirements for LTE highlight that the network should be optimized for UE at low mobile speed (0 to 15 km/h) whilst still supporting high mobile speed (15 to 120 km/h). It was observed that in the face of speed changes, RTT remains stable and no significant change in throughput was observed.
Time of Day
Time of day was not found to have significant impact on the observed results .
Monday 20 August 2012
TCPtrace - An introduction
What is TCPtrace ?
 | ||||
| Wireshark wins over TCPtrace on GUI |
So TCPtrace takes the output file from TCPdump as an input and it then outputs useful information and graphs.
How do I get TCPtrace ?
I downloaded it from the Ubuntu repositories using the typical 'sudo apt-get install tcptrace'. If this is not possible you can download it from here.How do I input a TCPdump file to TCPtrace ?
You can call TCPtrace with a TCPdump file using 'tcptrace <my-file>' where my-file is the name of the file outputted by TCPdump. For example you could do something like:$ sudo tcpdump -v -i wlan0 -w my_tcpdump_output -c 100
$ tcptrace my_tcpdump_output
The above will run TCPdump and create the output file called "my_tcpdump_output", this file is then passed as a argument to the TCPtrace tool
How do i interpret the basic TCPtrace output ?
The structure of the output is (in order from the top) :- 1st line returns the name of the TCPdump output file that TCPtrace is currently analyzing
- Then it printed the version of TCPtrace and then this version was last compiled
- The next line states the number of packets seen and the number of those which were TCP
- The following line gives the elapsed wallclock time, this is the time TCPtrace took to process the file and it then gives the average number of packets processed by TCPtrace per second
- The following line gives the trace file elasped time, this is the time between the capture of the first packet and the last packet in the file
- The sequential lines contain information on each TCP connection
- First it gives the address and ports of the two machines involved in the connection
- Then is the label given to this connection by TCPtrace is printed in parenthesis
- The number proceeding '>' is the number of packets seen in the first host to second host direction
- The number proceeding '<' is the number of packets seen in the second host to the first host direction
- Then the connection is labelled with '(complete)' or '(reset)', with the connection being labelled as complete if SYN and FIN packets were seen
When using TCPdump, you can see more detailed output using the '-v' option but with TCPtrace you can see more detailed output using the '-l' option.
When adding options to TCPtrace, you need to ensure the you place the extra options before the name of the input file and after the tool name.
When viewing the output from the long mode (when -l is the option) then all information is labelled. I'm now going to explain each label given in long output (warning .. this might take a while):
Packets and ACKS
- total packets - number of packets sent in that specific direction
- ack pkts sent - number of ACKs sent in that direction
- pure acks sent - number of ACK sent without data and the SYN,FIN&RST not set
- sack pkts sent - number of selective ACKs sent in that direction
- dsack pkts sent - number of duplicate selective ACKs sent in that direction
- max sack blks/ack - maximum number of selective ACK blocks seen in any selective ACK packet
- unique bytes sent - total number of bytes sent excluding retransmittions
- actual data pkts - total number of packet with at least 1 byte of TCP payload
- actual data bytes - total number of bytes seen including retransmittion
- rexmt data pkts - total number of packets that where retransmittions
- rexmt data bytes - total number of bytes of data that where retransmittions
- zxnd probe pkts - total number of window probe packets seen
- zxnd probe bytes - total number of bytes sent in window probe packets
- outoforder pkts - number of packets that arrived out of order
- pushed data pkts - number of packets with the PUSH bit set, this means that the buffered data should be sent to the receiving application
- SYN/FIN pkts sent - number of packets with SYN or FIN bits set
etc... (sorry I hate leaving things half done, but I really wanted to move on, its in my to-do list)
How do I get RTT (Round-Trip-Time) from TCPtrace ?
TCPtrace will generate statistics on RRT when using with the opinions '-r' and '-l'. This will give data on RRT including the number of RTT samples found, RTT minimum,RTT maximum, RTT average, RTT standard deviation, RTT from TCP's hand shake. The same data is then available again for full-sized RTT samples only.Monday 6 August 2012
TCPdump - An introduction
As per usual, if you find any mistakes, concepts that could be explained better or have something to add, then please comment below and I'll correct it.
What is TCPdump ?
TCPdump is a command line tool that is used to look at packets on the network. TCPdump does not generate the packets itself but instead it analyzes the packets generated by other applications and from this, it can determine network behaviour and performance.
Despite being called TCPdump, you can choose from a range of protocols including IP,ARP, TCP and UDP.
How do I get TCPdump ?
From Ubuntu, I simply got it via 'sudo apt-get install tcpdump'. Otherwise the public repo is located here, along side some useful information and the manual pages.
You can also check the dependencies of TCPdump using 'apt-cache depends tcpdump'. The output that this returns for me is:
Depends: libc6
Depends: libpcap0.8
Depends: libssl1.0.0
Suggests: apparmor
Conflicts: tcpdump:i386
If you get the package for the public repo, then you need will extract the content of the .tar.gz file using 'tar -zxf tcpdump-4.3.0.tar.gz' after using cd to move to the directory that you downloaded the file to, then install the program.
So I've got it.. but how do I use it ?
To start with use 'sudo tcpdump' to output to the terminal, this gives the standard output of TCPdump, using the default arguments since you have not passed TCPdump any arguments yet. To the output will be printed to the terminal, until you stop the program using Ctrl-C.
Don't be scared ... we will now begin the break this output down into understandable section (if you not scared then try 'sudo tcpdump -v' or 'sudo tcpdump -vv' for a verbose output).
Don't be scared ... we will now begin the break this output down into understandable section (if you not scared then try 'sudo tcpdump -v' or 'sudo tcpdump -vv' for a verbose output).
Changing network interface
When I'm using tcpdump, it listened on eth0 by default, but you can change this. Entering 'sudo tcpdump -D' will return a list of network interfaces that you can choose between. For me this returns:
1.eth0
2.wlan0
3.usbmon1 (USB bus number 1)
4.usbmon2 (USB bus number 2)
5.any (Pseudo-device that captures on all interfaces)
6.lo
The first item in the list is the Ethernet port, the 2nd is the Wireless card and the 3,4,5th is self explanatory. The 6th is the loopback virtual interface (more information here on wikipedia)
You can then change the interface, by calling TCPdump using 'sudo tcpdump -i <name-of-interface> '. For most of my work, I use the local Wi-Fi so can change the interface from Ethernet to wireless using 'sudo tcpdump -i wlan0'.
1.eth0
2.wlan0
3.usbmon1 (USB bus number 1)
4.usbmon2 (USB bus number 2)
5.any (Pseudo-device that captures on all interfaces)
6.lo
The first item in the list is the Ethernet port, the 2nd is the Wireless card and the 3,4,5th is self explanatory. The 6th is the loopback virtual interface (more information here on wikipedia)
You can then change the interface, by calling TCPdump using 'sudo tcpdump -i <name-of-interface> '. For most of my work, I use the local Wi-Fi so can change the interface from Ethernet to wireless using 'sudo tcpdump -i wlan0'.
Creating a file to store the tcpdump arguments
So far, we have only added one argument when we call TCPdump, but there will be a lot more to come. We can save these arguments, which filter the output of TCPdump to a file and then use the file when we call TCPdump.
Create the file using your favourite text editors (mine in gedit at the moment) and then pass the file to TCP dump using 'sudo tcpdump -F <my-file>'. The file dose not need any special file extension or layout.
Create the file using your favourite text editors (mine in gedit at the moment) and then pass the file to TCP dump using 'sudo tcpdump -F <my-file>'. The file dose not need any special file extension or layout.
You may need to change the file permissions so that TCPdump can extract the file. You can view the file premissions using 'ls -l <my-file>'. The 'ls' part is command line tool to list the files in a directory and the argument -l means use long format so the file permissions will be included. The file 10 characters are the file permissions, they are decoded as follows:
- 'd' means this is a directory and '-' means this is a file
- 'r' means that the file owner can read the file, '-' means they can't
- 'w' means that the file owner can write to the file, '-' means they can't
- 'x' means that the file owner can execute the file, '-' means they can't. If this is a directory then 'x' means that the owner can list the files in directory, '-' means they can't.
- 'r' means that the file group can read the file, '-' means they can't
- 'w' means that the file group can write to the file, '-' means they can't
- 'x' means that the file group can execute the file, '-' means they can't. If this is a directory then 'x' means that the owner can list the files in directory, '-' means they can't.
- 'r' means that everyone else can read the file, '-' means they can't
- 'w' means that everyone else can write to the file, '-' means they can't
- 'x' means that everyone else can execute the file, '-' means they can't. If this is a directory then 'x' means that the owner can list the files in directory, '-' means they can't.
For me, the default file permissions are '-rw-rw-r--'. This therefore means that the file owner doesn’t have permission to execute the file. This can be changed using 'chmod u+x <my-file>'. The command line tool is chmod, this is used for changing file permissions, 'u' means we are considering the users permissions, '+' means we are granting a permission and 'x' means we can considering the execute permission. Now, if I redo 'ls -l <my-file>' the new result is -rwxrw-r--.
Sending the output to file
You can send the output of tcpdump to a file instead of printing it to the terminal using 'sudo tcpdump -w <my-file>'. This output is not saved as a plain text so you can't just read it in a text editor instead you need to use a special program to read the file and extract information from it. You can do this using tcpdump by 'tcpdump -r <my-file>'. Alternatively you can open it using wireshark, launch wireshark and using File>Open.
Filtering information by protocol
To filter packets by protocol, add the name of protocol to the arguments of tcpdump so use something like 'sudo tcpdump <name-of-protocol>'. The main protocols that I'm likely are use are IP, ARP, TCP and UDP but others are available, see the man pages for a full list
Filtering information by direction to different hosts
To filter packets by a direction and host, add the direction and then the host name. Possible direction options are src, dst, src or dst, src and dst. You specify the host using its IP address. You can use the logical operators not, or and and. For example, you can look up the local IP address of your machine using 'ifconfig <name-of-interface>'. If you don't specify the name of the interface, then all interfaces will be listed. Now if you only want to view incoming traffic you can use 'sudo udpdump dst <ip-address>'.
Change the level of detail
Compared to the level of detail provided by the standard query, The detail can be reduced using '-q' for quiet output or increased using '-v' for verbose, '-vv' for more verbose and '-vvv' for even more verbose.
The opinion '-t' means do not print timestamp on each line and the option '-a' allows you to display each packet in ASCII or '-x' to display in hex or '-X' to display in hex and ASCII
View IP address instead of domain names
You can stop the translation of IP address to domain names using the '-n' opinion and you can stop the translation of port number too using the '-nn' opinion
Sources of Info
Wikipedia article, which contains very little information
Official public repo, including the man pages and FAQ
Useful online tutorial at openmaniak, this site also has good tutorials on networking tools that I've previously covered including wireshark, OpenVPN, Iperf and ping.
Sunday 5 August 2012
A new week, a new approach
I'm changing my approach from using tools like Iperf and ping to collect network data and then using my java program to analyses the output to writing the scripts for myself and working from the ground up.
It is new territory for me so it really exciting but also a bit daunting. Along side this new work, I now have just 7 days left to prepare for the Google Android Development Camp in London. The next week is looking like its going busy but interesting and by the end I hope to have overcome a steep learning curve.
PLAN FOR MONDAY
I'll update you again soon...
It is new territory for me so it really exciting but also a bit daunting. Along side this new work, I now have just 7 days left to prepare for the Google Android Development Camp in London. The next week is looking like its going busy but interesting and by the end I hope to have overcome a steep learning curve.
PLAN FOR MONDAY
I'll update you again soon...
Wednesday 1 August 2012
Data Collection for Latency, Goodput & Jitter - Demo Pt 5.2
Everyone who I have spoken to about my work since yesterday, has asked me the same question. Why are you writing this in Java ? The answer is that I am going on the Google European Android Development Camp in a few weeks so I am using Java were possible in my work so that I can get familiar with the basics again.
To ensure that the server (my laptop) and the client (my android phone) can address each other I ensure that they are behind the same NAT so that private IP addresses can be used. This ensures that both devices can initialize an connect with the other
My Java code for analyzing and comparing Signpost Diagnostic Application to the results generated by Iperf and Ping, requires the following files:
Once you have generated all of these files then you can run my program SignpostOutputAnalysis.java found here which should output the average true and estimated latency, goodput and jitter.
The code in SignpostOutputAnalysis.java is still incomplete and untested, I also have not yet tested my instructions for generating the correct files at the correct locations for the java code to be ran. I will be doing this testing next...
To ensure that the server (my laptop) and the client (my android phone) can address each other I ensure that they are behind the same NAT so that private IP addresses can be used. This ensures that both devices can initialize an connect with the other
My Java code for analyzing and comparing Signpost Diagnostic Application to the results generated by Iperf and Ping, requires the following files:
- SignOutput.txt - this is generated by the Signpost Diagnostic Application, It can be generated by:
- Hard coding the server IP address into the application code, found here
- Uninstall previous versions of the application and connect the android phone via USB
- Within eclipse, load the application onto the android phone
- cd into Downloads/android-sdk-linux/platform-tools
- In another terminal, cd into the location of the demo server and start server using ./server.native
- use "./adb logcat-s SIGPST >> SignOutput.txt"
- The location of the file SignOutput.txt is now Downloads/android-sdk-linux/platform-tools, copy the file to /TestingSignpostAppOutput
- IperfOutput.txt - this is generated by a remote shell on the Android phone, it can be generated by:
- (In terminal 1) cd into Downloads/android-sdk-linux/platform-tools
- connect android phone via USB
- use "./adb shell" to start a remote shell
- in another terminal (terminal 2), cd into /TestingSignpostAppOutput so output file is loaded straight into correct directory
- (In terminal 2) use iperf -s -u >> IperfOutput.txt
- (In terminal 1) use iperf -u -c {insert laptop IP}
- PingDownstreamOutput.txt - this is generated by the server sending pings to the client (the android phone), it can be generated by:
- cd into /TestingSignpostAppOutput so output file is loaded straight into correct directory
- Run a bash script containing the following:
- #!/bin/bash
for i in {1..10}
do
ping {insert phone IP} -c 10 -n -q >> PingDownstreamOutput.txt;
done - PingUpstreamOutput.txt - this is generated by the client (android phone) sending pings to the server, it can be generated by:
- cd into Downloads/android-sdk-linux/platform-tools
- Save a bash script (lets call it pingUp) containing the following
- #!/bin/bash
for i in {1..10}
do
ping {insert laptop IP} -c 10 -n -q >> PingUpstreamOutput.txt;
done - Connect android phone via USB and copy script to SD cards using "./adb push pingUp / sdcard/ "
- use "./adb shell" to start a remote shell
- In the remote shell cd into /sdcard
- In the remote shell run the script using ./pingUp
- exit the remote shell
- cd into /TestingSignpostAppOutput
- copy the PingUpstreamOutput.txt file to the laptop from the android phone using "./adb pull /sdcard/PingUpstreamOutput.txt
Once you have generated all of these files then you can run my program SignpostOutputAnalysis.java found here which should output the average true and estimated latency, goodput and jitter.
The code in SignpostOutputAnalysis.java is still incomplete and untested, I also have not yet tested my instructions for generating the correct files at the correct locations for the java code to be ran. I will be doing this testing next...
Tuesday 31 July 2012
Data Collection for Latency - Demo Pt 5.1
Now its time for me to collect data from the Signpost Diagnostics Application and Iperf to see if there is a statistically significant difference between the Latency and Goodput data from the two applications. [Jitter will soon be added to the application so I will need to go back the test that]
THE PLAN...
Part A - Data from Signpost Application
AND THE REALITY...
I quick come to realize that the above is stage is useless. It will correctly install an application from the .apk file but I need to be able to edit the code of the application so that I can set the client IP address at a later stage
I connect both the client and server to the same Wi-FI network so that the client will be able in initiate an connection with the server
As per usual, to start up the OCaml Server, I will do
I uninstalled the last Signpost Application, change the IP address in the code and load onto phone, via Eclipse using the run as dialogue.
Pressing start on the Android application triggers a connection to be initiated with the server, this is successful as the sever outputs a link detailing the android phones name, IP address and port number used.
I can view the logs for this application live, using:
To filter the output of logcat, I use the arguments -s SIGPST, this gives me a log output such as:
I/SIGPST ( 1520): Received Latency Upstream: 25500
I/SIGPST ( 1520): Received Goodput Downstream: 4026
I/SIGPST ( 1520): Received Latency Downstream: 6008
I/SIGPST ( 1520): Received Goodput Downstream: 1262925
I/SIGPST ( 1520): Received Latency Upstream: 49000
I/SIGPST ( 1520): Received Goodput Downstream: 4787
I/SIGPST ( 1520): Received Latency Downstream: 10376
I/SIGPST ( 1520): Received Latency Upstream: 50000
I/SIGPST ( 1520): Received Goodput Downstream: 2735
I/SIGPST ( 1520): Received Latency Downstream: 17004
I/SIGPST ( 1520): Received Latency Upstream: 46500
I/SIGPST ( 1520): Received Goodput Downstream: 4461
I/SIGPST ( 1520): Received Latency Downstream: 4463
I/SIGPST ( 1520): Received Goodput Downstream: 799720
I/SIGPST ( 1520): Received Latency Upstream: 51000
I/SIGPST ( 1520): Received Goodput Downstream: 3684
I/SIGPST ( 1520): Received Latency Downstream: 6875
I/SIGPST ( 1520): Received Latency Upstream: 307000
I/SIGPST ( 1520): Received Goodput Downstream: 4670
I/SIGPST ( 1520): Received Latency Downstream: 8749
I/SIGPST ( 1520): Received Latency Upstream: 16500
I/SIGPST ( 1520): Received Goodput Downstream: 2652
I/SIGPST ( 1520): Received Latency Downstream: 6795
I/SIGPST ( 1520): Received Goodput Downstream: 24060150
I/SIGPST ( 1520): Received Latency Upstream: 72500
I/SIGPST ( 1520): Received Goodput Downstream: 4190
I/SIGPST ( 1520): Received Latency Downstream: 6809
A quick code inspection, highlights that the latency values are divided by 1000 to be converted into secs, which means that the latency values here are in ms
I will look at how to convert this to a more convenient form in part 3
I can append the output to the file Signoutput.txt using ">> Signoutput.txt"
I have added a sample of Signoutput.txt here
24 packets transmitted, 24 received, 0% packet loss, time 23070ms
rtt min/avg/max/mdev = 35.614/90.183/297.302/51.944 ms
The latency will be half the RRT so here it is 45.0915 ms
Then repeating the test from the server to the client and i get an output such as
43 packets transmitted, 40 received, 6% packet loss, time 42070ms
rtt min/avg/max/mdev = 33.371/93.812/241.825/45.579 ms
So the latency is similar to before at 46.906 ms
Now I want to collect the data from the ping output, so I wrote the following bash script to do this:
#!/bin/bash
for i in {1..10}
do
ping 192.168.14.47 -c 10 -n -q >> Pingoutput.txt;
done
This sents 100 pings, in 10 sets of 10 and sent the results of the 10 test to a file called Pingoutput.txt
The first line "#!/bin/bash" mean that this is a bash script, the ping argument "-c 10" means said 10 pings and "-q" means quiet output. >> means append to file.
I've added a sample of Pingoutput.txt here
Part C
To analysis the output files, I have written the Java code here. Currently this code is untested
THE PLAN...
Part A - Data from Signpost Application
- Connect the android phone (the client) and laptop (the server) to Internet in such a way that they are behind the same NAT box, so the client can initiate a connection with the server from a private IP address
- Set up the OCaml Server running on my laptop
- Change in client IP address in the client code and load onto android phone
- Running client with server, collecting data whilst viewing output from log files
- Workout how to send pings bothways
- Workout how to use ping to get RRT bothways
- Collect some data on latency
- Extract sufficient data from both methods
- Get data from both method into a suitable format
- If required, convert units for latency so all data in same units
- Compare data
- Answer the Question: Is there a significant difference between the data collected by Ping (assumed to be accurate) and the Signpost Application
AND THE REALITY...
Part A
I've just been send the most recent version of the Signpost Diagnostics Application as a .apk file. I uninstall the old version of the application using the Android GUI and install the new version using:
cd /android-sdk-linux/platform-tools
./adb install ~/Downloads/SigcommDemoAndroid.apk
I quick come to realize that the above is stage is useless. It will correctly install an application from the .apk file but I need to be able to edit the code of the application so that I can set the client IP address at a later stage
I connect both the client and server to the same Wi-FI network so that the client will be able in initiate an connection with the server
As per usual, to start up the OCaml Server, I will do
cd Downloads/sebastian-SignpostDemo-53ebd3e/SignpostServerOCaml/
./server.native
./server.native
I uninstalled the last Signpost Application, change the IP address in the code and load onto phone, via Eclipse using the run as dialogue.
Pressing start on the Android application triggers a connection to be initiated with the server, this is successful as the sever outputs a link detailing the android phones name, IP address and port number used.
I can view the logs for this application live, using:
cd Downloads/android-sdk-linux/platform-tools
./adb logcat-s SIGPSTTo filter the output of logcat, I use the arguments -s SIGPST, this gives me a log output such as:
I/SIGPST ( 1520): Received Latency Upstream: 25500
I/SIGPST ( 1520): Received Goodput Downstream: 4026
I/SIGPST ( 1520): Received Latency Downstream: 6008
I/SIGPST ( 1520): Received Goodput Downstream: 1262925
I/SIGPST ( 1520): Received Latency Upstream: 49000
I/SIGPST ( 1520): Received Goodput Downstream: 4787
I/SIGPST ( 1520): Received Latency Downstream: 10376
I/SIGPST ( 1520): Received Latency Upstream: 50000
I/SIGPST ( 1520): Received Goodput Downstream: 2735
I/SIGPST ( 1520): Received Latency Downstream: 17004
I/SIGPST ( 1520): Received Latency Upstream: 46500
I/SIGPST ( 1520): Received Goodput Downstream: 4461
I/SIGPST ( 1520): Received Latency Downstream: 4463
I/SIGPST ( 1520): Received Goodput Downstream: 799720
I/SIGPST ( 1520): Received Latency Upstream: 51000
I/SIGPST ( 1520): Received Goodput Downstream: 3684
I/SIGPST ( 1520): Received Latency Downstream: 6875
I/SIGPST ( 1520): Received Latency Upstream: 307000
I/SIGPST ( 1520): Received Goodput Downstream: 4670
I/SIGPST ( 1520): Received Latency Downstream: 8749
I/SIGPST ( 1520): Received Latency Upstream: 16500
I/SIGPST ( 1520): Received Goodput Downstream: 2652
I/SIGPST ( 1520): Received Latency Downstream: 6795
I/SIGPST ( 1520): Received Goodput Downstream: 24060150
I/SIGPST ( 1520): Received Latency Upstream: 72500
I/SIGPST ( 1520): Received Goodput Downstream: 4190
I/SIGPST ( 1520): Received Latency Downstream: 6809
A quick code inspection, highlights that the latency values are divided by 1000 to be converted into secs, which means that the latency values here are in ms
I will look at how to convert this to a more convenient form in part 3
I can append the output to the file Signoutput.txt using ">> Signoutput.txt"
I have added a sample of Signoutput.txt here
Part B
I open a remote shell on the android phone using ./adb shell, I then use the ping unix commend from the client to the server, and i get an output such as24 packets transmitted, 24 received, 0% packet loss, time 23070ms
rtt min/avg/max/mdev = 35.614/90.183/297.302/51.944 ms
The latency will be half the RRT so here it is 45.0915 ms
Then repeating the test from the server to the client and i get an output such as
43 packets transmitted, 40 received, 6% packet loss, time 42070ms
rtt min/avg/max/mdev = 33.371/93.812/241.825/45.579 ms
So the latency is similar to before at 46.906 ms
Now I want to collect the data from the ping output, so I wrote the following bash script to do this:
#!/bin/bash
for i in {1..10}
do
ping 192.168.14.47 -c 10 -n -q >> Pingoutput.txt;
done
This sents 100 pings, in 10 sets of 10 and sent the results of the 10 test to a file called Pingoutput.txt
The first line "#!/bin/bash" mean that this is a bash script, the ping argument "-c 10" means said 10 pings and "-q" means quiet output. >> means append to file.
I've added a sample of Pingoutput.txt here
Part C
To analysis the output files, I have written the Java code here. Currently this code is untested
Subscribe to:
Posts (Atom)