Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Friday, 11 October 2013

Apache Thrift Tutorial

1 comment

Introduction

       Apache thrift is a tool developed at Facebook which is used for automatically creating remote servers in Remote Procedure Call (RPC) in different languages. Thrift means economical, since it is saving time and cost for creating remote servers in RPC.  Apache thrift support many different languages . Apache thrift won't create client, you require to create the client program manually. Apache thrift is an Interface Definition language (IDL) used to create services in a different language.

Installation 

On Ubuntu:
       You can install all dependencies by giving only one single command as shown below
sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev 

OR

       If there is an error in above command please check due to which dependency error is occurring.
  • First check whether your g++ version is greater or equal to 4.2 you may check the version with the command given below
    g++ --version   (This should be >= 4.2)
  • Install lex and yacc using command given below
    sudo apt-get install byacc flex
  • Install boost library using command given below
    sudo apt-get install libboost-all-dev
  • Install autoconf which is require for building the source
    sudo apt-get install autoconf
  • Install  automake
    sudo apt-get install automake
  • Install libtool
    sudo apt-get install libtool
  • install libssl-dev
     sudo apt-get install libssl-dev
  • Install pkg-config
    sudo apt-get install pkg-config
  • Install libevent-dev
    sudo apt-get install libevent-dev
  • Install bison
    sudo apt-get install bison

       Now download the apache thrift and extract it. After extracting open terminal and go to the path where thrift is extracted and give the following commands
./configure
make
sudo make install
       You can check whether apache thrift is installed properly or not by giving command below. If you are able to see the version of thrift then your installation is completed successful.
thrift --version

Use

      As explain before we can create source codes for many languages without installation of that language, but to run created source code you will require to install those language. To create remote server you will require to create a file with extension thrift. This thrift file contain thrift interface description language we will explain this in short for detail refer link.

Lets create one example which performs basic arithmetic operations Thrift_IDL.thrift.

#this is comment
#!/usr/local/bin/thrift --gen cpp

namespace cpp Test #this indicates that given thrift is applicable to cpp 

service arithmetic_op  #create class arithmetic_opHandler : virtual public arithmetic_opIf with following public function
{       
  i32 add(1:i32 num1, 2:i32 num2),  #this create function with int32_t add(const int32_t num1, const int32_t num2)
  i32 sub(1:i32 num1, 2:i32 num2),  #this create function with int32_t sub(const int32_t num1, const int32_t num2) 
  i32 mul(1:i32 num1, 2:i32 num2),  #this create function with int32_t mul(const int32_t num1, const int32_t num2) 
  i32 div(1:i32 num1, 2:i32 num2),  #this create function with int32_t div(const int32_t num1, const int32_t num2)
}

service arithmetic_Demo      #this is second service for dispay
{
i32 display(1:i32 num1, 2:i32 num2)
}
To generate server for this file you require to give the command
thrift --gen cpp Thrift_IDL.thrift
       This command will generate one folder with name "gen-cpp" which contain 10 files as shown in the figure below. This number of file and the format of files changes as the language changes, here we are considering the files for cpp.
Apache Thrift
  
       Since we don't define any struct or enum declared in thrift file, the file Thrift_IDL_types.h, Thrift_IDL_types.cpp are only initialized but not used.

Modify Functions declared in Services(.thrift file):
       Functions declared in the services of thrift file are only initialized by thrift. You require to add the contain to perform some action you require.
       We will take example of arithmetic_op service: thrift will create arithmetic_op_server.skeleton.cpp file which contain main function and thrift will create arithmetic_opHandler class which contain constructor and the number of functions declared in the thrift file as shown below.
class arithmetic_opHandler : virtual public arithmetic_opIf {
 public:
  arithmetic_opHandler() {
    // Your initialization goes here
  }

  int32_t add(const int32_t num1, const int32_t num2) {
    // Your implementation goes here
    printf("add\n");
  }

  int32_t sub(const int32_t num1, const int32_t num2) {
    // Your implementation goes here
    printf("sub\n");
  }

  int32_t mul(const int32_t num1, const int32_t num2) {
    // Your implementation goes here
    printf("mul\n");
  }

  int32_t div(const int32_t num1, const int32_t num2) {
    // Your implementation goes here
    printf("div\n");
  }
};
        You can add logic of addition, subtraction, multiplication and division in the functions add(), sub(), mul(), div() respectively. This functions are taking two inputs from client and printing the function name, instead of this you can add the logic of arithmetic operation.
Compiling:
  • g++ -I/usr/local/include/thrift -c arithmetic_Demo.cpp
  • g++ -I/usr/local/include/thrift -c arithmetic_Demo_server.skeleton.cpp
  • g++ -I/usr/local/include/thrift -c arithmetic_op.cpp
  • g++ -I/usr/local/include/thrift -c arithmetic_op_server.skeleton.cpp
  • g++ -I/usr/local/include/thrift -c Thrift_IDL_constants.cpp
  • g++ -I/usr/local/include/thrift -c Thrift_IDL_types.cpp
To compile all files in one stroke you can use the command given below
g++ -I/usr/local/include/thrift -c *.cpp
Here *cpp represent all files with extension .cpp.
Explanation:
      -I: option which is used by gcc compiler to locate the header files. In this compilation we are suggesting the gcc compiler to search for the header files in the "/usr/local/include/thrift/" folder also.
      Since it is only compilation gcc compiler won't link the thrift libraries which eliminates the requirement of including thrift library with -lthrift.
      -c: option in gcc compiler is used to create object files if you don't provide -o followed by your custom name of object file it will create object file with the same name of your c or cpp file with .o extension.

Linking:
       Since there are two services in our example we get two Service_name_server_skeleton.cpp files which contain main function in it. Due to this we require to link this two service_skeleton.cpp files separately, which will create two executables with two different service.
       You can use the command given below for linking the services

Link "arithmetic_op" service:
 
g++ -L/usr/local/lib/ arithmetic_op.o arithmetic_op_server.skeleton.o Thrift_IDL_constants.o Thrift_IDL_types.o -lthrift -o arithmetic_op

Link "arithmetic_Demo" service:

g++ -L/usr/local/lib/ arithmetic_Demo.o arithmetic_Demo_server.skeleton.o Thrift_IDL_constants.o Thrift_IDL_types.o -lthrift -o arithmetic_Demo

Explanation:
-L: This option is used when you want compiler to link with libraries present in directory other than LD_LIBRARY_PATH environment variable. Here we have used to link with thrift library named "libthrift.a" or "libthrift.so". Here .a represent static library where as .so represent dynamic library.

-o: This option is used to give custom name to your executable by default it is a.out
Read More...

Friday, 13 September 2013

Pass multiple arguments in pthread_create

Leave a Comment
Pthreads

        Posix thread normally known as pthread. It is an API to create threads on Unix like operating system such as GNU/Linux, MAC etc. Windows doesn't provide support for this API. In Windows some third party softwares are available which can be used for writing multithread application using pthread, pthreads-w32 is one of this.
        To create thread pthread uses pthread_create function with syntax given below.

int pthread_create(pthread_t *thread, const pthread_attr_t *attr, void *(*start_routine) (void *), void *arg);
        This function creates a new thread and that thread will execute the function specified as third parameter with arguments specified in fourth parameter. There is no provision to transfer more than one argument to the function which is called by pthread_create.
        If you want to transfer multiple arguments then you require to create a structure with arguments you require to pass to the function and then by transferring this structure you can pass as many arguments as you want to transfer. The program pthread.c below is one such example of passing multiple parameters


#include"pthread.h"
#include"stdlib.h"
#include"stdio.h"

struct test
{
 int var1,var2;
 float var3;
};

typedef struct test struct1;

void* fun1(void* a)
{
 struct1 *b;
 b=(struct1*)a;
 b->var3=b->var1+b->var2;
 printf("\nOperation successful the sum is %f\n",b->var3);
}

int main()
{
 pthread_t th1,th2;
 struct1 *a;
 a=(struct1 *)malloc(sizeof(struct1));
 a->var1=10;
 a->var2=20;
 pthread_create(&th1,NULL,fun1,(void*) a);
 pthread_join(th1,NULL);
 return 0;
}


Read More...

Friday, 6 September 2013

Strace : Moniter system calls

Leave a Comment

Introduction

Strace is a small useful debugging tool which traces all the system calls and signals used by  the program. Internally Strace uses ptrace system call to trace the system calls executed by the program. This list of system calls is useful to get clear idea, how program is interacting with the kernel.

Installation:

In Ubuntu you can install Strace using 
sudo apt-get install strace
In CentOS you can install by
yum install strace

OR 
Download the Strace-4.8.tar.xz
tar xvJf strace-4.8.tar.xz
cd strace-4.8/
./configure
make
make install        (as root or sudoer user)

Use: 

  1. strace who        (Default it will print all the system call and signals)
    execve("/usr/bin/who", ["who"], [/* 42 vars */]) = 0
    brk(0)                                 = 0x1568000
    access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
    mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2e366b1000
    access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
    open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
    ..
    ..
  2. strace -o output.txt who (-o is used to write the result of strace utility in the file)
    strace who > output.txt
    This will write the result of executing command here it is who, in the output.txt file not the system calls that are traced by the strace command.
  3. strace -c who (-c is used to calculate total number of times the system call is appeared without giving the details of each system call)
  4. strace -C who (-C is used to calculate total number of times the system call is appeared with giving the details of each system call)
  5. strace -e write who (-e option is used to detect specific system calls provided out of all)
    write(1, "krishna  tty7         2013-09-06"..., 44krishna  tty7         2013-09-06 09:36 (:0)
    ) = 44
    write(1, "krishna  pts/0        2013-09-06"..., 46krishna  pts/0        2013-09-06 12:47 (:0.0)
    ) = 46
    write(1, "krishna  pts/1        2013-09-06"..., 46krishna  pts/1        2013-09-06 11:54 (:0.0)
    ) = 46
    +++ exited with 0 +++

    strace -e trace=read,write who (We can use this command to detect multiple system calls as we have perform above example for only one)
  6. We can also get the traces of currently running (executing) process
    strace -p 2700 (here 2700 is the process id of the currently running job)
Read More...

Sunday, 21 July 2013

Disable usb drives in Windows and Linux

Leave a Comment
There are many reasons due to which you may want to disable the USB drive such as virus attacks in windows, in exam to stop students from using USB drives. We can use following method to stop users other than administrator from using USB drive.

Disable USB Drive in Window

Step 1:  click Start->Run
Step 2:  in Run windows type regedit. Regedit is a graphical view for all configuration information about the inbuilt and installed software in windows.

Step 3: In regedit select the path as   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\USBSTOR


Step 4: In this select the Start file after this select Hexadecimal base, by default it is blank now enter 4 in the value data field and click OK and close regedit and your USB drives gets disabled. If you want to enable USB drive again then open this file, delete the value and keep it blank.


Disable USB Drives in Linux

Step 1: First check the kernel release of your current kernel OS because one operating system may have many versions of kernel. When we update Linux it keeps older version of kernel we can find current kernel version using uname -r command use  
kernel_version=$(uname -r) 
command so that you can use kernel_version variable for our next purpose.

Step 2: Enter following command using root privileges
mv /lib/modules/kernel_version/kernel/drivers/usb/storage
/lib/modules/Kernel_version/kernel/drivers/usb/storage_old
this command is used to change the name of storage directory to storage_old which disable the USB drive. If you want to access the USB drive again, change the name of this storage_old directory to storage again using root privileges.
Read More...

Tuesday, 11 June 2013

No valid sudoers sources found quitting ubuntu

1 comment
This error happen due to change in permission of sudoers file.
Pkexec command is used to execute command as another user. Older versions of Ubuntu don't support this command. If you are able to use pkexec command then you are not require to reboot and go to recovery mode use following commands

pkexec chown -R root:root /etc/sudoers.d
pkexec chmod u=rwx,g=rx,o=rx /etc/sudoers.d/
pkexec chmod u=r,g=r,o= /etc/sudoers.d/*


If you are running on old versions which could not support pkexec then you require to use recovery mode.
You can go into recovery mode by selecting grub menu ending with recovery mode option ->root -> give root password
after this enter command

fdisk -l (gives all detail disk partition)

Since in ubuntu recovery mode is read only you will require to remount in read-write mode using following command

mount -o remount,rw /dev/sda(drive containing installation i.e root)
now enter above commands without pkexec as below

chown -R root:root /etc/sudoers.d
chmod u=rwx,g=rx,o=rx /etc/sudoers.d/
chmod u=r,g=r,o= /etc/sudoers.d/*


now reboot and you will be able to use sudo or su command

Read More...

Sunday, 26 May 2013

passwd command not working in single user mode

Leave a Comment
If your passwd command is not working in single user mode and you want access of machine then first remove the password of root or any user already present as explained below. After removing password, login to that user without password and change password for the user by using passwd command.
You can remove the password by editing shadow file present in /etc/ folder with file name shadow
Shadow file format is as below
root:$1$CQoPk7Zh$370xDLmeGD9m4aF/ciIlC.:14425:0:99999:7:::

There are 6 fields separated by ':'. To remove password delete second field present in line with first field showing user name it should look like
root::14425:0:99999:7:::

This file is read only so after changing file you require to give command as wq! to save changes in vi editor (since in single user mode other editors won't work) this will change your file. If this is not working then you have to change the permissions of the shadow file. To change permission use chmod 777 shadow command.

Check the file is changed, if it is changed then you can restart by command reboot.



Read More...

Sunday, 28 April 2013

Install Linux on External Hard Disk or Pen Drive

Leave a Comment

 1. Why to install OS on external hard-disk or pen drive?

        The main reason is mobility. Carrying laptop every where is more difficult than carrying pen drive or External hard-disk. If you have any system available in your office or college, then you can simply do your work by booting through external hard-disk and after completing your work you can carry it to your home and restart the work by booting through external hard-disk.

2. Requirement

  • External hard-disk with at least free space which fulfills the requirement of your OS.
  • Bootable CD/DVD or bootable pen drive.
  • Please check system hardware (which you want to access using OS installed on external hard-disk) whether it supports booting from USB or not. Many recent hardware support booting from external hard-disk or pen drive.

3. How to install Linux on external hard-disk?

       First create one free partition which we can use for installing Linux as follow.
1. Right click on My Computer in Windows OS then click on manage ->Storage ->Disk Management Then you will get screen as shown in image below.
       After this Right click on Disk 1 and click on Shrink Volume and follow the steps to create new disk drive.
resize hard drive

       After creating free partition start installation as we do normal installation by restarting PC. You have to plug in external hard-disk or pen drive throughout  installation. Select installation type as "something else".


       First create boot sector with 500 MB which is require to store grub.

Second create root drive with size you want to use for Linux.

Third you must have to create swap partition which is used for storing swap files in hard disk which is faster than other drive.

The main Part of your installation is selecting proper drive to install boot loader. You must select the drive partition which is used for the external hard disk. You can get list of drives by giving command " fdisk -l " in Linux or by going to computer management in Windows as shown earlier. In Linux for every new hard disk or pen drive it appends letter from a-z to sd*, sda is for first disk sdb is for second and so on. You can select proper drive as shown in following image.

Now you can start installation by clicking install now.

After completing installation restart your system go to boot menu
Read More...

Thursday, 25 April 2013

Why Linux ?

2 comments


1. What is Linux.

      Linux is a open source operating system first release developed by Linus Torvalds in 5 Oct 1991. Linux kernel is developed in C language with GCC compiler. Linux is developed as open source Operating System (OS), so that any one can modify the kernel and get modified new OS as per the requirement. One most popular example is Android the widely used mobile OS which is developed by using Linux kernel. Almost all super computers run on Linux operating system due to its great reliability, security and open source features than Windows.

2. Why Linux is not so popular

  • Linux is not user friendly as windows.
  • Installing and using different software in windows is much simpler and easier than Linux.

3. Why to use Linux  

  • Virus won't affect.
  • Linux is available for free which make you free from using pirated copies of windows or purchasing its costly license copy.
  • It is open source that mean you can modify Linux kernel and develop your own OS.
  • Security of Linux is far stronger than windows which get affected by viruses and many other attacks.
  • Linux is very reliable OS which can run (continue) years without hanging. 

4.Types of Linux operating System

Widely we can categories Linux into three part
  • Red Hat Based (Red Hat, CentOS, Fedora etc.)
  • Debian Based (Ubuntu )
  • Slackware (SUSE) 

  4.1 Red Hat Based (CentOS, Fedora)

      This type  of OS is developed by rebuilding Red Hat Linux. This type of Linux uses RPM package manager. RPM refers to the .rpm package software used by Red Hat based OS. This are server end OS. This type of OS are very stable than other OS, but using this OS is difficult compared to other. Most famous OS in this type are CentOS, Fedora, Red Hat etc. Red Hat Enterprise linux is Commercial OS which is not free, but the redistribution of this OS such as CentOS, Fedora is freely available.

  4.2 Debian Based (Ubuntu ) 

     Debian is an association of people which makes open source OS. This type of OS are more user friendly than any other Linux OS, but using this type of OS at server end is not good. This OS contain many software packages around 29,000 present in official Debian distribution. Most famous OS in this type of distribution is Ubuntu.

  4.3 Slackware (SUSE)

    Slackware is earliest OS which modified Linux kernel so that Linux becomes more user friendly and stable. Most popular OS in this type is different SUSE distribution.

5.  Which Linux Flavour to choose. 

      In my opinion one who is working in IT or CSE domain can use Red Hat based OS. In Red Hat based OS one can use CentOS since it is very stable and easy to use compared to other Red Hat based OS.
      General students can use Ubuntu or openSUSE which is very user friendly.

Read More...