Search Posts on Binpipe Blog
Installing HAProxy Loadbalancer in CentOS, Redhat or Ubuntu
HAProxy is fast evolving as a dependable Loadbalancing solution. Here I have outlined the steps to install HAProxy to build your own software load balancer.
Installing HAProxy
For most distributions you can install haproxy using your distribution's package manager. For example, to install on Debian or Ubuntu, run:
sudo aptitude install haproxy
CentOS 5
We will need to set up access to the EPEL software repository to download haproxy on CentOS 5. Run the commands:
[root@LB01 ~]# rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
[root@LB01 ~]# yum -y install haproxy
CentOS 6
We will need to set up access to the EPEL software repository to download haproxy on CentOS 6, but the address for the RPM is different from CentOS 5. Run the commands:
[root@LB01 ~]# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-6.noarch.rpm
[root@LB01 ~]# yum -y install haproxy
Install a base config
Once installed backup the HAProxy config file and download the managed cloud config:
[root@LB01 ~]# cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak
[root@LB01 ~]# wget http://c818095.r95.cf2.rackcdn.com/haproxy.cfg -O /etc/haproxy/haproxy.cfg
chkconfig haproxy on
Configuring HAProxy
Configuring HAProxy can only come after you have your web heads configured as you will need to utilize their 10.x service net IP address's. The reason we use the service net is because the customer will not be charged for bandwidth overage, and the service net is also faster in terms of throughput as shown in the chart at the top.
Editing /etc/haproxy/haproxy.cfg - There are a number of items that need to be changed in order to get HAProxy functional. These will be outlined below. Keep in mind you need to edit these values to reflect the server's IP's.
First and foremost change
listen webfarm 0.0.0.0:80
to
listen webfarm 127.0.0.1:80
Edit 127.0.0.1 to reflect your server's eth0 or public IP.
Now you can add your web servers. In the following you will want to replace the 10.0.0.X IP address with that of the eth1 or private IP address of web servers"
server WWW1 10.0.0.1:80 check # Active in rotation
server WWW2 10.0.0.2:80 check # Active in rotation
server WWW3 10.0.0.3:80 check # Active in rotation
server WWW4 10.0.0.4:80 check backup # Not active "sorry server" - this one comes live if all web heads are down
Above is an example of what a four server config would look like. Once you have completed this portion you can then start HAProxy and start serving pages(assuming your web servers are ready).
service haproxy start
Below is the default configuration template for haproxy.cfg:
#global options
global
#logging is designed to work with syslog facility's due to chrooted environment
#log loghost local0 info - By default this is commented out
#chroot directory
chroot /usr/share/haproxy
#user/group id
uid 99
gid 99
#running mode
daemon
defaults
#HTTP Log format
mode http
#number of connection retries for the session
retries 3
#try another webhead if retry fails
option redispatch
#session settings - max connections, and session timeout values
maxconn 10000
contimeout 10000
clitimeout 50000
srvtimeout 50000
#Define your farm
#listen webfarm 0.0.0.0:80 - Pass only HTTP traffic and bind to port 80
listen webfarm 0.0.0.0:80
#HTTP Log format
mode http
#stats uri /haproxy - results in http://<load balancer ip>/haproxy (shows load balancer stats)
stats uri /haproxy
#balance roundrobin - Typical Round Robin
#balance leastconn - Least Connections
#balance static-rr - Static Round Robin - Same as round robin, but weights have no effect
balance roundrobin
#cookie <COOKIENAME> prefix - Used for cookie-based persistence
cookie webpool insert
#option httpclose - http connection closing
option httpclose
#option forwardfor - best stated as "Enable insertion of the X-Forwarded-For header to requests sent to the web heads" aka send EU IP
option forwardfor
#Web Heads (Examples)
#server WEB1 10.0.0.1:80 check - passes http traffic to this server and checks if its alive
#server WEB1 10.0.0.1:80 check port 81 - same as above but checks port 81 to see if its alive (helps to remove servers from rotation)
#server WEB1 10.0.0.1:80 check port 81 weight 100 - same as the above with weight specification (weights 1-256 / higher number higher weight)
#server WEB1 10.0.0.1:80 check backup - defines this server as a backup for the other web heads
#Working Example: *USE THIS HOSTNAME FORMAT*
server WWW1 10.0.0.1:80 cookie webpool_WWW1 check port 81 # Active in rotation
server WWW2 10.0.0.2:80 cookie webpool_WWW2 check port 81 # Active in rotation
server WWW3 10.0.0.3:80 check # Active in rotation
server WWW4 10.0.0.4:80 check backup # Not active "sorry server" - this one comes live if all web heads are down
#SSL farm example
#listen https 0.0.0.0:443
# mode tcp
# server WEB1 10.0.0.1:443 check
Session Persistence with SSL
If the you wish to also balance SSL traffic, you will need to set the balance mode to "source" This setting takes a hash of the client's IP address and the number of servers in rotation, sending traffic from one IP address to the same web server consistently. The persistence will be reset if the number of servers is changed.:
listen https 0.0.0.0:443
mode tcp
balance source
server WEB1 10.0.0.1:443 check
Postfix as SMTP Relay
If you need to install Postfix and use it as a SMTP relay, you can follow these steps below:
Install Postfix and cyrus-sasl with your application manager of choice. If you're compiling from source, be sure to make Postfix with the -DUSE_SASL_AUTH flag for SASL support and -DUSE_TLS for TLS support.
$ yum install postfix cyrus-sasl$ /etc/init.d/sendmail stop$ chkconfig --del sendmailEdit /etc/postfix/main.cf
# Set this to your server's fully qualified domain name.# If you don't have a internet domain name,# use the default or your email addy's domain - it'll keep# postfix from generating warnings all the time in the logsmydomain = local.domainmyhostname = host.local.domain# Set this to your email provider's smtp server.# A lot of ISP's (ie. Cox) block the default port 25# for home users to prevent spamming. So we'll use port 80relayhost = yourisp.smtp.servername:80smtpd_sasl_auth_enable = yessmtpd_sasl_path = smtpdsmtp_sasl_password_maps = hash:/etc/postfix/sasl_passwdsmtp_sasl_type = cyrussmtp_sasl_auth_enable = yes# optional: necessary if email provider uses load balancing and# forwards emails to another smtp server# for delivery (ie: smtp.yahoo.com --> smtp.phx.1.yahoo.com)smtp_cname_overrides_servername = no# optional: necessary if email provider# requires passwords sent in clear textsmtp_sasl_security_options = noanonymousAdd the following line to /etc/postfix/sasl_passwd
yourisp.smtp.servername:80 username:passwordGenerate a postfix lookup table from the previous file
$ postmap hash:/etc/postfix/sasl_passwd$ postmap -q yourisp.smtp.servername:80 /etc/postfix/sasl_passwd$ chmod 600 /etc/postfix/sasl_passwd$ chmod 600 /etc/postfix/sasl_passwd.db$ chkconfig --add postfix$ /etc/init.d/postfix start$ sendmail email@example.comPostfix is good to go.
--The Below Steps are specifics. Please ignore if nor required.--If you're attempting to relay mail using Gmail, then it will be necessary to use TLS with Postfix. You'll have to point Postfix at your server's trusted CA root certificate bundle. If that is the case then read below or else ignore.
First, double-check that Postfix was configured with SSL support (ie. ldd should return at least one line starting with libssl):
$ whereis -b postfixpostfix: /usr/sbin/postfix /etc/postfix /usr/libexec/postfix$ ldd /usr/sbin/postfix...libssl.so.6 => /lib/libssl.so.6 (0x00111000)...$ locate ca-bundle.crt/etc/pki/tls/certs/ca-bundle.crtrelayhost = smtp.gmail.com:587# your FQDN, or default value belowmydomain = local.domain# your local machine name, or default value belowmyhostname = host.local.domainmyorigin = $myhostname# SASLsmtpd_sasl_path = smtpdsmtp_sasl_password_maps = hash:/etc/postfix/sasl_passwdsmtp_sasl_type = cyrussmtp_sasl_auth_enable = yessmtp_sasl_security_options = noanonymous# TLSsmtp_sasl_tls_security_options = noanonymoussmtp_use_tls = yessmtp_tls_CAfile = /path/to/your/ca-bundle.crtsmtp_sasl_tls_security_options = noanonymoussmtp.gmail.com:587 username:password$ postmap hash:/etc/postfix/sasl_passwd$ chmod 600 /etc/postfix/sasl_passwd$ chmod 600 /etc/postfix/sasl_passwd.db$ postfix reload$ sendmail email@example.comTest relay thru Gmail
If you need to do some debugging please read below or ignore:Monitor postfix mail log in a separate session with the following command
$ tail -f /var/log/maillog(Authentication failed: cannot SASL authenticate to server ...: no mechanism available)smtp_sasl_security_options = noanonymous553 Sorry, that domain isn't in my list of allowed rcpthosts. (in reply to RCPT TO command)Load Balancing Techniques
Load balancing is a term that describes a method to distribute incoming socket connections to different servers. It’s not distributed computing, where jobs are broken up into a series of sub-jobs, so each server does a fraction of the overall work. It’s not that at all. Rather, incoming socket connections are spread out to different servers. Each incoming connection will communicate with the node it was delegated to, and the entire interaction will occur there. Each node is not aware of the other nodes existence.
Why do you need load balancing?
Simple answer: Scalability and Redundancy.
Scalability
There are 3 well known ways:
DNS based
This is where it gets fun, if you’re a technology enthusiast. If your budget doesn’t allow a load balancing appliance, or if you just like doing things yourself, software based load balancing is for you. You can turn a Linux server into your own load balancing appliance. Presumably, you could also use a Windows server, maybe even a Mac, but this article doesn’t cover those. For RHEL based, the “piranha” package provides Linux Virtual Server (LVS) and piranha (an LVS management tool – web based gui). Just “yum install piranha” and you’ll have everything you need to get started. Other softwares include BalanceNG (commercial) and a basic freeware counterpart balance.
This was super simple to use. Just download, run the program. There are a few basic input parameters, and you can be load balancing in no time. This is a no frills binary program. There are no configuration files, no startup/shutdown programs, no logging or reporting. But it does have a nifty console that you can get runtime statistics from. You could create your own tools around “balance” to monitor and gather statistics.
LVS and piranha on RHEL or CentOS
piranha is a gui that makes configuring Linux Virtual Server (LVS) easy. Here are some of the virtual server scheduling features:
Why do you need load balancing?
Simple answer: Scalability and Redundancy.
Scalability
If your application becomes busy, resource limits, such as bandwidth, cpu, memory, disk space, disk I/O, and more may reach its limits. In order to remedy such problem, you have two options: scale up, or scale out. Load balancing is a scale out technique. Rather than increasing server resources, you add cost effective, commodity servers, creating a “cluster” of servers that perform the same task. Scaling out is more cost effective, because commodity level hardware provides the most bang for the buck. High end super computers come at a premium, and can be avoided in many cases.Redundancy
Servers crash, this is the rule, not the exception. Your architecture should be devised in a way to reduce or eliminate single points of failure (SPOF). Load balancing a cluster of servers that perform the same role provides room for a server to be taken out manually for maintenance tasks, without taking down the system. You can also withstand a server crashing. This is called High Availability, or HA for short. Load balancing is a tactic that assists with High Availability, but is not High Availability by itself. To achieve high availability, you need automated monitoring that checks the status of the applications in your cluster, and automates taking servers out of rotation, in response to failure detected. These tools are often bundled into Load Balancing software and appliances, but sometimes need to be programmed independently.How to perform load balancing?
There are 3 well known ways:
- DNS based
- Hardware based
- Software based
DNS based
This is also known as round robin DNS. You can inject multiple A records for the same hostname. This creates a random distribution – requests for the hostname will receive the list in a random order. If you wish to weight it (say serverA can take 2x the number of requests that serverB can), you can simply add more A records for a particular IP.Hardware based
There are many commercial vendors out there selling appliances to perform load balancing.Software based
Hardware based load balancing is the best way to go, if you have budget for it. These appliances provide the latest features, with little fuss.
This is where it gets fun, if you’re a technology enthusiast. If your budget doesn’t allow a load balancing appliance, or if you just like doing things yourself, software based load balancing is for you. You can turn a Linux server into your own load balancing appliance. Presumably, you could also use a Windows server, maybe even a Mac, but this article doesn’t cover those. For RHEL based, the “piranha” package provides Linux Virtual Server (LVS) and piranha (an LVS management tool – web based gui). Just “yum install piranha” and you’ll have everything you need to get started. Other softwares include BalanceNG (commercial) and a basic freeware counterpart balance.
This was super simple to use. Just download, run the program. There are a few basic input parameters, and you can be load balancing in no time. This is a no frills binary program. There are no configuration files, no startup/shutdown programs, no logging or reporting. But it does have a nifty console that you can get runtime statistics from. You could create your own tools around “balance” to monitor and gather statistics.
LVS and piranha on RHEL or CentOS
piranha is a gui that makes configuring Linux Virtual Server (LVS) easy. Here are some of the virtual server scheduling features:
- Round robin
- Weighted least-connections
- Weighted round robin
- Least-connection
- Locality-Based Least-Connection Scheduling
- Locality-Based Least-Connection Scheduling (R)
- Destination Hash Scheduling
- Source Hash Scheduling
Directory Tree in Linux
If you want to get the hierarchy of the directory in a tree format there is the tree command for that. But here I have put the command in a small loop so that the output becomes more informative with the directory and file sizes also.
Try this command out in the bash shell and see the output:
# for i in $(ls -d */); do tree $i ; done
Note: If the directory structure is too long to caontain it in a screen you can use the following to save it to a file:
# for i in $(ls -d */); do tree $i ; done > result.txt
Try this command out in the bash shell and see the output:
# for i in $(ls -d */); do tree $i ; done
Note: If the directory structure is too long to caontain it in a screen you can use the following to save it to a file:
# for i in $(ls -d */); do tree $i ; done > result.txt
Find Total Number of Sub-directories and Files in a Directory
We have often wondered how many sub-directories and files are there recursively under a directory in Linux. Sometimes, we may also need to calculate the number so that we don't cross inode count in shared hosting servers.
The following code snippet will allow you to get the count of files & directories inside a folder. Please run this command after doing a 'cd' to inside the folder.
find $DIR -exec stat -c '%F' {} \; | sort | uniq -c | sort -rn
If you want a tree view of the directories with the count you can use this:
for i in $(ls -d */); do tree $i ; done > result.txt
or try this one below:
for i in $(ls -d */); do tree $i | grep -v \\-\\-\ ; done
Quite simple but effective one-liners aren't they? :-)
The following code snippet will allow you to get the count of files & directories inside a folder. Please run this command after doing a 'cd' to inside the folder.
find $DIR -exec stat -c '%F' {} \; | sort | uniq -c | sort -rn
If you want a tree view of the directories with the count you can use this:
for i in $(ls -d */); do tree $i ; done > result.txt
or try this one below:
for i in $(ls -d */); do tree $i | grep -v \\-\\-\ ; done
Quite simple but effective one-liners aren't they? :-)
Configure NTP to Synchronize Server Time

The following steps are sufficient to make the NTP Sync.
Login as the root user
Type the following command to install NTP:
# yum install ntpTurn on service:
# chkconfig ntpd onSynchronize the system clock with 0.pool.ntp.org server:
# ntpdate pool.ntp.orgStart the NTP:
# /etc/init.d/ntpd start
Passwordless SSH Login
You want to access computer A securely from computer B without having to enter a password. This technique is very useful for automation, where you don't want to put passwords into scripts.
On computer B login as the user you want to use to access computer A, and generate yourself a pair of public private keys by issuing the following command, and simply press return to the prompts to use an
empty passphrase. On computer B login as the user you want to use to access computer A, and generate yourself a pair of public private keys by issuing the following command, and simply press return to the prompts to use an empty passphrase.
ssh-keygen -t rsa
This command created a public/private key pair in your home directory under the directory of ~/.ssh. Your public key is stored in the file
~/.ssh/id_rsa.pub
Now we need to create an ~/.ssh directory on the remote computer B if it doesn't exist, under the username we're going to access it with, substitute the username your going to use in the following. you'll need
at this stage to enter the password for username@B
> ssh username@B mkdir -p .ssh
username@B's password:
Now use ssh to push your public key to computer B via the following command, you'll again need to enter username@b's password
> cat .ssh/id_rsa.pub | ssh username@B 'cat >> .ssh/authorized_keys'
username@B's password:
That's it !
From now on entering
From now on entering
ssh username@b
Will get you logged is as username on computer B without the need to
enter a password.
Analyse LAMP Server Slowness
Analysis of LAMP server slowness can be done basically by following these four steps:
- Track MySQL Queries Taking over a Second
By default, mysql will log queries which take 10 seconds or longer. Depending on your installation, these may or may not be logged to a file. Certainly, if you have queries showing up here, at 10+ seconds, you should investigate them. In most cases, I expect that you'll be investigating performace before the point at which you have numerous queries taking over ten seconds. On a production site, the vast majority of your queries should be returning in significantly under a second.
To lower your slow query threshold and enable logging, you'll want to modify yourmy.cnf(often/etc/mysql/my.cnf), and ensure that you have settings like this:
set-variable=long_query_time=1 log-slow-queries=/var/log/mysql/slow_query.log
You may need to touch that file, and ensured that it is owned and writable by your mysql process. When you restart mysql, look in the mysql error log (often/var/log/mysql/mysql.err) for any errors along the lines of "Could not use /var/log/slow_query.log for logging (error 13)"_. If you see these, create the @slowquery.log@ and set it's ownership to that of your mysql user.
Now, depending on the state of your system,slow_query.logwill begin to accumulate queries. The actual format of the slow log is bit verbose, butmysqldumpslow, a perl script included with most mysql installations can parse it and produce some more meaningful output. It will take various integers in your queries (a user_id, thread_id, etc) and generalize them, so you can locate types, instead of specific queries.> mysqldumpslow -t=10 -s=t /var/log/slow_query.log Reading mysql slow query log from /var/log/slow_query.log Count: 46 Time=80.46s (3701s) Lock=0.00s (0s) Rows=512311 (117447821), bob[bob]@[10.0.0.32] SELECT * FROM forum_posts Count: 26 Time=68.26s (1775s) Lock=0.00s (0s) Rows=426 (117447821), bob[bob]@[10.0.0.32] SELECT * FROM forum_posts WHERE thread=N Count: 120 Time=3.52s (422s) Lock=0.63s (76s) Rows=58.0 (6960), bob[bob]@[10.0.0.32] SELECT authors FROM forum_posts WHERE lastpost > N ...
The next step is analyzing this, likely throwing each of these into an EXPLAIN query (or asking yourself why you are selecting every row in the forum_posts table), adding some indexes, and rewriting some code. The scope of this article is finding the bottlenecks… fixing them is left as an exercise for the reader. - Monitor PHP Memory Usage & Log Apache Delivery Times
Out of the box, your apache install is likely using the NCSA extended/combined log format. You're going to take this format and add to pieces of data to it. The first will be the memory used by PHP during the rendering of each page. The second will be the time that apache spends delivering this page. Both of these values will be tacked onto the end of the log format. Many log processing scripts will ignore fields added onto the end of the line, so adding them here is least likely to break things.
Unless you've mucked with it, your httpd.conf likely has lines like this:LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common CustomLog /usr/local/apache/logs/access_log common </pre>Whichever log format is being used, this is the name at the end of theCustomLogdirective, you're going to make a copy of thatLogFormat, give it a name like "commondebug", and switch the CustomLog directive to use this format. The fields you will be adding are: *%T– The time taken to server the request in seconds *%{mod_php_memory_usage}n– Memory used by PHP in bytesLogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T %{mod_php_memory_usage}n" combineddebug LogFormat "%h %l %u %t \"%r\" %>s %b %T %{mod_php_memory_usage}n" commondebug CustomLog /usr/local/apache/logs/access_log commondebug </pre>At this point, you'll be collecting some great data in you apache logs. You can get some good information with some quick shell magic, like so:> awk '{printf("%07d\t%d\t%s\n", $(NF), $(NF-1), $7)}' access_log | sed 's/\?.*//' | sort -g -k1 0001232 0 /baz.php 0001232 0 /bar.php ... 1712160 0 /foo.php 1717640 0 /foo.php 1907800 0 /foo.php 2010840 0 /foo.php </pre>Replace,-k1with-k2to sort by the delivery times. Keep in mind, delivery times will include the time to send the bytes over the network — http clients can do screwy things, and you'll occasionally see anomalous data including 120+ second connects where the client just stopped accepting packets, but didn't close the socket. From here, you'll want to examine each of the memory-hogging scripts, and anything which is consitently long-running. - Log PHP Errors
This is one that is obvious, but easy to miss. Many sites disable the display of errors, viaphp.inion their production servers. (This is a good idea, preventing the revelation of any inappopriate information to end users.)
You'll be modifying yourphp.inito include lines such as these:error_reporting = E_ALL & ~E_NOTICE ; Show all errors except for notices display_errors = Off ; Do not print out errors (as a part of the HTML script) log_errors = On ; Log errors into a log file error_log = "/var/log/php_error.log" ; log errors to specified file
By itself, this isn't likely to pinpoint any performance issues, but it may help you locate other issues. Although, if are really bad and you may see a slew of "maximum execution time of XX seconds exceeded" in various pages. You are more likely to see errors which correlate to long-running or memory-consuming scripts and queries identified earlier. - Take Snapshots at an OS Level
This is the area that makes the average developer wish they had a sysadmin on call. Unfortunately, for many small sites and projects, the developer is forced to wear that hat as well. At some point, if usage gets high enough, no amount of redesign or optimization will be enough to stretch your hardware further. So the question will be, what is the bottleneck? RAM, IO, CPU?
On this topic, your options are pretty endless. Linux provides endless tools for monitoring resources, and different people will recommend different ones. Something incredibly simple, like below can be pretty informative.#!/usr/bin/perl my $URL_TO_TEST = 'http://test.test/test.test'; my $THRESHOLD = 2; open(my $log, '>>status_log.txt'); while (1) { my $start = time(); `curl "$URL_TO_TEST" > /dev/null`; my $took = time() - $start; if ($took > $THRESHOLD) { print $log "$took seconds load ::"; print $log `date`; print $log "\n\n"; print $log `vmstat -a`; print $log "\n\n"; print $log `uptime`; print $log "\n\n"; print $log `ps awux`; print $log "\n\n"; print $log `mysqladmin -hHOST -uUSER -pPASSWORD processlist`; print $log "----------------------------------------------\n\n"; } sleep(30); }
Handy Mysql Commands
These are a few handy Mysql commands I have always used. You will find them useful particularly if you are a newbie. Please add more of them in the comments so that it becomes a really useful thread.
Please note that below when you see # it means from the Linux/Unix shell. When you see mysql> it means from a MySQL prompt after logging
into MySQL.
To login (from unix shell) use -h only if needed.
# [mysql dir]/bin/mysql -h hostname -u root -p
Create a database on the sql server.
mysql> create database [databasename];
List all databases on the sql server.
mysql> show databases;
Switch to a database.
mysql> use [db name];
To see all the tables in the db.
mysql> show tables;
To see database's field formats.
mysql> describe [table name];
To delete a db.
mysql> drop database [database name];
To delete a table.
mysql> drop table [table name];
Show all data in a table.
mysql> SELECT * FROM [table name];
Returns the columns and column information pertaining to the designated table.
mysql> show columns from [table name];
Show certain selected rows with the value "whatever".
mysql> SELECT * FROM [table name] WHERE [field name] = "whatever";
Show all records containing the name "Bob" AND the phone number '3444444'.
mysql> SELECT * FROM [table name] WHERE name = "Bob" AND phone_number = '3444444';
Show all records not containing the name "Bob" AND the phone number '3444444' order by the phone_number field.
mysql> SELECT * FROM [table name] WHERE name != "Bob" AND phone_number = '3444444' order by phone_number;
Show all records starting with the letters 'bob' AND the phone number '3444444'.
mysql> SELECT * FROM [table name] WHERE name like "Bob%" AND phone_number = '3444444';
Show all records starting with the letters 'bob' AND the phone number '3444444' limit to records 1 through 5.
mysql> SELECT * FROM [table name] WHERE name like "Bob%" AND phone_number = '3444444' limit 1,5;
Use a regular expression to find records. Use "REGEXP BINARY" to force case-sensitivity. This finds any record beginning with a.
mysql> SELECT * FROM [table name] WHERE rec RLIKE "^a";
Show unique records.
mysql> SELECT DISTINCT [column name] FROM [table name];
Show selected records sorted in an ascending (asc) or descending (desc).
mysql> SELECT [col1],[col2] FROM [table name] ORDER BY [col2] DESC;
Return number of rows.
mysql> SELECT COUNT(*) FROM [table name];
Sum column.
mysql> SELECT SUM(*) FROM [table name];
Join tables on common columns.
mysql> select lookup.illustrationid, lookup.personid,person.birthday from lookup left join person on
lookup.personid=person.personid=statement to join birthday in person table with primary illustration id;
Creating a new user. Login as root. Switch to the MySQL db. Make the user. Update privs.
# mysql -u root -p
mysql> use mysql;
mysql> INSERT INTO user (Host,User,Password) VALUES('%','username',PASSWORD('password'));
mysql> flush privileges;
mysql> use mysql;
mysql> INSERT INTO user (Host,User,Password) VALUES('%','username',PASSWORD('password'));
mysql> flush privileges;
Change a users password from unix shell.
# [mysql dir]/bin/mysqladmin -u username -h hostname.blah.org -p password 'new-password'
Change a users password from MySQL prompt. Login as root. Set the password. Update privs.
# mysql -u root -p
mysql> SET PASSWORD FOR 'user'@'hostname' = PASSWORD('passwordhere');
mysql> flush privileges;
mysql> SET PASSWORD FOR 'user'@'hostname' = PASSWORD('passwordhere');
mysql> flush privileges;
Recover a MySQL root password. Stop the MySQL server process. Start again with no grant tables. Login to MySQL as root. Set new password. Exit MySQL and restart MySQL server.
# /etc/init.d/mysql stop
# mysqld_safe --skip-grant-tables &
# mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("newrootpassword") where User='root';
mysql> flush privileges;
mysql> quit
# /etc/init.d/mysql stop
# /etc/init.d/mysql start
# mysqld_safe --skip-grant-tables &
# mysql -u root
mysql> use mysql;
mysql> update user set password=PASSWORD("newrootpassword") where User='root';
mysql> flush privileges;
mysql> quit
# /etc/init.d/mysql stop
# /etc/init.d/mysql start
Set a root password if there is on root password.
# mysqladmin -u root password newpassword
Update a root password.
# mysqladmin -u root -p oldpassword newpassword
Allow the user "tom" to connect to the server from localhost using the password "passwd". Login as root. Switch to the MySQL db. Give privs. Update privs.
# mysql -u root -p
mysql> use mysql;
mysql> grant usage on *.* to tom@localhost identified by 'passwd';
mysql> flush privileges;
mysql> use mysql;
mysql> grant usage on *.* to tom@localhost identified by 'passwd';
mysql> flush privileges;
Give user privilages for a db. Login as root. Switch to the MySQL db. Grant privs. Update privs.
# mysql -u root -p
mysql> use mysql;
mysql> INSERT INTO db (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv) VALUES ('%','databasename','username','Y','Y','Y','Y','Y','N');
mysql> flush privileges;
or
mysql> grant all privileges on databasename.* to username@localhost;
mysql> flush privileges;
mysql> use mysql;
mysql> INSERT INTO db (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv) VALUES ('%','databasename','username','Y','Y','Y','Y','Y','N');
mysql> flush privileges;
or
mysql> grant all privileges on databasename.* to username@localhost;
mysql> flush privileges;
To update info already in a table.
mysql> UPDATE [table name] SET Select_priv = 'Y',Insert_priv = 'Y',Update_priv = 'Y' where [field name] = 'user';
Delete a row(s) from a table.
mysql> DELETE from [table name] where [field name] = 'whatever';
Update database permissions/privilages.
mysql> flush privileges;
Delete a column.
mysql> alter table [table name] drop column [column name];
Add a new column to db.
mysql> alter table [table name] add column [new column name] varchar (20);
Change column name.
mysql> alter table [table name] change [old column name] [new column name] varchar (50);
Make a unique column so you get no dupes.
mysql> alter table [table name] add unique ([column name]);
Make a column bigger.
mysql> alter table [table name] modify [column name] VARCHAR(3);
Delete unique from table.
mysql> alter table [table name] drop index [colmn name];
Load a CSV file into a table.
mysql> LOAD DATA INFILE '/tmp/filename.csv'
replace INTO TABLE [table name] FIELDS TERMINATED BY ',' LINES
TERMINATED BY '\n' (field1,field2,field3);
Dump all databases for backup. Backup file is sql commands to recreate all db's.
# [mysql dir]/bin/mysqldump -u root -ppassword --opt >/tmp/alldatabases.sql
or # [mysql dir]/bin/mysqldump -u root -ppassword --all-databases >/tmp/alldatabases.sql
or # [mysql dir]/bin/mysqldump -u root -ppassword --all-databases >/tmp/alldatabases.sql
Dump one database for backup.
# [mysql dir]/bin/mysqldump -u username -ppassword --databases databasename >/tmp/databasename.sql
Dump a table from a database.
# [mysql dir]/bin/mysqldump -c -u username -ppassword databasename tablename > /tmp/databasename.tablename.sql
Restore database (or database table) from backup.
# [mysql dir]/bin/mysql -u username -ppassword databasename < /tmp/databasename.sql
Restore all databases from backup with --all-databases option.
Create Table Example 1.
mysql> CREATE TABLE [table name] (firstname
VARCHAR(20), middleinitial VARCHAR(3), lastname VARCHAR(35),suffix
VARCHAR(3),officeid VARCHAR(10),userid
VARCHAR(15),username VARCHAR(8),email VARCHAR(35),phone VARCHAR(25),
groups VARCHAR(15),datestamp DATE,timestamp time,pgpemail VARCHAR(255));
Create Table Example 2.
mysql> create table [table name] (personid int(50)
not null auto_increment primary key,firstname varchar(35),middlename
varchar(50),lastnamevarchar(50) default
'bato');
MYSQL Statements and clauses
ALTER DATABASE
ALTER TABLE
ALTER VIEW
ANALYZE TABLE
BACKUP TABLE
CACHE INDEX
CHANGE MASTER TO
CHECK TABLE
CHECKSUM TABLE
COMMIT
CREATE DATABASE
CREATE INDEX
CREATE TABLE
CREATE VIEW
DELETE
DESCRIBE
DO
DROP DATABASE
DROP INDEX
DROP TABLE
DROP USER
DROP VIEW
EXPLAIN
FLUSH
GRANT
HANDLER
INSERT
JOIN
KILL
LOAD DATA FROM MASTER
LOAD DATA INFILE
LOAD INDEX INTO CACHE
LOAD TABLE...FROM MASTER
LOCK TABLES
OPTIMIZE TABLE
PURGE MASTER LOGS
RENAME TABLE
REPAIR TABLE
REPLACE
RESET
RESET MASTER
RESET SLAVE
RESTORE TABLE
REVOKE
ROLLBACK
ROLLBACK TO SAVEPOINT
SAVEPOINT
SELECT
SET
SET PASSWORD
SET SQL_LOG_BIN
SET TRANSACTION
SHOW BINLOG EVENTS
SHOW CHARACTER SET
SHOW COLLATION
SHOW COLUMNS
SHOW CREATE DATABASE
SHOW CREATE TABLE
SHOW CREATE VIEW
SHOW DATABASES
SHOW ENGINES
SHOW ERRORS
SHOW GRANTS
SHOW INDEX
SHOW INNODB STATUS
SHOW LOGS
SHOW MASTER LOGS
SHOW MASTER STATUS
SHOW PRIVILEGES
SHOW PROCESSLIST
SHOW SLAVE HOSTS
SHOW SLAVE STATUS
SHOW STATUS
SHOW TABLE STATUS
SHOW TABLES
SHOW VARIABLES
SHOW WARNINGS
START SLAVE
START TRANSACTION
STOP SLAVE
TRUNCATE TABLE
UNION
UNLOCK TABLES
USE
String Functions
AES_DECRYPT
AES_ENCRYPT
ASCII
BIN
BINARY
BIT_LENGTH
CHAR
CHAR_LENGTH
CHARACTER_LENGTH
COMPRESS
CONCAT
CONCAT_WS
CONV
DECODE
DES_DECRYPT
DES_ENCRYPT
ELT
ENCODE
ENCRYPT
EXPORT_SET
FIELD
FIND_IN_SET
HEX
INET_ATON
INET_NTOA
INSERT
INSTR
LCASE
LEFT
LENGTH
LOAD_FILE
LOCATE
LOWER
LPAD
LTRIM
MAKE_SET
MATCH AGAINST
MD5
MID
OCT
OCTET_LENGTH
OLD_PASSWORD
ORD
PASSWORD
POSITION
QUOTE
REPEAT
REPLACE
REVERSE
RIGHT
RPAD
RTRIM
SHA
SHA1
SOUNDEX
SPACE
STRCMP
SUBSTRING
SUBSTRING_INDEX
TRIM
UCASE
UNCOMPRESS
UNCOMPRESSED_LENGTH
UNHEX
UPPER
Date and Time Functions
ADDDATE
ADDTIME
CONVERT_TZ
CURDATE
CURRENT_DATE
CURRENT_TIME
CURRENT_TIMESTAMP
CURTIME
DATE
DATE_ADD
DATE_FORMAT
DATE_SUB
DATEDIFF
DAY
DAYNAME
DAYOFMONTH
DAYOFWEEK
DAYOFYEAR
EXTRACT
FROM_DAYS
FROM_UNIXTIME
GET_FORMAT
HOUR
LAST_DAY
LOCALTIME
LOCALTIMESTAMP
MAKEDATE
MAKETIME
MICROSECOND
MINUTE
MONTH
MONTHNAME
NOW
PERIOD_ADD
PERIOD_DIFF
QUARTER
SEC_TO_TIME
SECOND
STR_TO_DATE
SUBDATE
SUBTIME
SYSDATE
TIME
TIMEDIFF
TIMESTAMP
TIMESTAMPDIFF
TIMESTAMPADD
TIME_FORMAT
TIME_TO_SEC
TO_DAYS
UNIX_TIMESTAMP
UTC_DATE
UTC_TIME
UTC_TIMESTAMP
WEEK
WEEKDAY
WEEKOFYEAR
YEAR
YEARWEEK
Mathematical and Aggregate Functions
ABS
ACOS
ASIN
ATAN
ATAN2
AVG
BIT_AND
BIT_OR
BIT_XOR
CEIL
CEILING
COS
COT
COUNT
CRC32
DEGREES
EXP
FLOOR
FORMAT
GREATEST
GROUP_CONCAT
LEAST
LN
LOG
LOG2
LOG10
MAX
MIN
MOD
PI
POW
POWER
RADIANS
RAND
ROUND
SIGN
SIN
SQRT
STD
STDDEV
SUM
TAN
TRUNCATE
VARIANCE
Flow Control Functions
CASE
IF
IFNULL
NULLIF
Command-Line Utilities
comp_err
isamchk
make_binary_distribution
msql2mysql
my_print_defaults
myisamchk
myisamlog
myisampack
mysqlaccess
mysqladmin
mysqlbinlog
mysqlbug
mysqlcheck
mysqldump
mysqldumpslow
mysqlhotcopy
mysqlimport
mysqlshow
perror
Perl API - using functions and methods built into the Perl DBI with MySQL
available_drivers
begin_work
bind_col
bind_columns
bind_param
bind_param_array
bind_param_inout
can
clone
column_info
commit
connect
connect_cached
data_sources
disconnect
do
dump_results
err
errstr
execute
execute_array
execute_for_fetch
fetch
fetchall_arrayref
fetchall_hashref
fetchrow_array
fetchrow_arrayref
fetchrow_hashref
finish
foreign_key_info
func
get_info
installed_versions
last_insert_id
looks_like_number
neat
neat_list
parse_dsn
parse_trace_flag
parse_trace_flags
ping
prepare
prepare_cached
primary_key
primary_key_info
quote
quote_identifier
rollback
rows
selectall_arrayref
selectall_hashref
selectcol_arrayref
selectrow_array
selectrow_arrayref
selectrow_hashref
set_err
state
table_info
table_info_all
tables
trace
trace_msg
type_info
type_info_all
Attributes for Handles
PHP API - using functions built into PHP with MySQL
mysql_affected_rows
mysql_change_user
mysql_client_encoding
mysql_close
mysql_connect
mysql_create_db
mysql_data_seek
mysql_db_name
mysql_db_query
mysql_drop_db
mysql_errno
mysql_error
mysql_escape_string
mysql_fetch_array
mysql_fetch_assoc
mysql_fetch_field
mysql_fetch_lengths
mysql_fetch_object
mysql_fetch_row
mysql_field_flags
mysql_field_len
mysql_field_name
mysql_field_seek
mysql_field_table
mysql_field_type
mysql_free_result
mysql_get_client_info
mysql_get_host_info
mysql_get_proto_info
mysql_get_server_info
mysql_info
mysql_insert_id
mysql_list_dbs
mysql_list_fields
mysql_list_processes
mysql_list_tables
mysql_num_fields
mysql_num_rows
mysql_pconnect
mysql_ping
mysql_query
mysql_real_escape_string
mysql_result
mysql_select_db
mysql_stat
mysql_tablename
mysql_thread_id
mysql_unbuffered_query
Subscribe to:
Posts (Atom)






