Linux Find and Replace with sed

Have you ever found yourself googling how to find and replace within a file from the linux command line? Me too. Here’s the answer that works for me: sed -i 's/search/replace/g' /path/to/file.txt

Rather than type that command all the time, I wrote a bash script to simplify things. BE WARNED! I know very little about bash scripting or the sed command, so this script probably only works for simple search and replace strings. I bet it would break if either string contained quotes or other reserved characters. That being said, it works like a charm for simple stuff. I named it esed (for easy sed).

Usage: esed search replace /path/to/file.txt

It’ll ask you to confirm the replace, then after a successful run it’ll show the contents of the file. I’ve found that useful for small files, but not so useful for big ones. You could always comment that line out if it annoys you.   :)

[note color=”#DDD”]Download
[/note]

Posted in Googling for Answers, Linux Tagged

Bash Script to Show All Cron Jobs by All Users

I googled this up the other day and it’s a winner for sure. The script was tagged as the best answer to “How do I list all cron jobs for all users?” over at stackoverflow.

[note color=”#DDD”]Download
[/note]

Posted in Googling for Answers, Linux Tagged ,

Recursively chmod only Directories or Files

This particular trick is invaluable, and I’m tired of googling it. You can read the original article here.

[note color=”#DDD”]Directories Only:
find . -type d -exec chmod 755 {} \;[/note] [note color=”#DDD”]Files Only:
find . -type f -exec chmod 644 {} \;[/note] And here are some other ones I found in the comments on the original blog:

[note color=”#DDD”]For files matching a pattern:
find . -type f -name '*.htm*' -exec chmod 644 {} \;[/note] [note color=”#DDD”]According to one commenter, this will work if you’re having trouble with directories or files with spaces in them:
find . -type d -exec chmod 755 \"{}\" \;[/note] [note color=”#DDD”]Supposedly this method runs faster (and solves the problem with spaces):
find . -type d -print0 |xargs -0 chmod 740
find . -type f -print0 |xargs -0 chmod 640[/note]

Posted in Googling for Answers, Linux Tagged ,

Getting this damn blog to look the way I wanted!

I don’t even know where to begin. First, finding a good theme to use as a starting point is HARD. Don’t let anyone tell ya different. Second, tweaking that theme (the right way) is not always easy.

After hours of theme searching I finally came across Grey Opaque by H.-Peter Pfeufer. This theme puts most other free themes to shame. It’s downright sexy! Next, I needed to customize it a bit. I know just enough about WordPress to know that you should make a child theme for your changes, rather than edit any of the original theme’s files. The reason being that when your theme gets an update, it could wipe out all of your customization. It’s not too difficult to do manually, but I happened across a plugin that makes it super easy.

After setting up my child theme, I was able to tweak the CSS to affect most of my changes. Here’s a handy tip: if you don’t already know the joy of the !important rule in CSS, get to know it before trying to style a child theme. To keep it short and under-explained, the !important rule will make sure that your CSS gets applied, even if the normal rules of CSS wouldn’t have done so. KEEP IN MIND THOUGH, that !important will make that rule absolutely unchangeable, even with javascript, as I found out later.

[note color=”#DDD”]When you just gots to have an orange border!
div#example { border: 2px solid orange !important; }[/note]

Here’s where things got a little crazy. I decided that I wanted to setup some of my post categories as pages. For example, I wanted a page on my site called Linux, to show all my posts in the Linux category. One of the themes I tried on before finding Grey Opaque had this feature, so I assumed it would be pretty easy to implement. Turns out it’s not. Well, that’s not entirely true either, it can be fairly simple, but not if you want the category page’s posts to match the rest of your theme.

The first thing I did was install a plugin called List category posts. It’s a pretty amazing plugin and I highly recommend trying it out if you have need of something like this. It has shortcode with tons of options for displaying the posts the way you want. It also has a template system for taking even more control of the look and feel. At first I just used CSS to bend it to my will, but I eventually did end up making my own template as well. I got things pretty darn close to where I wanted them, but the blog posts on the category pages still weren’t identical to the main blog page and I wasn’t having it.

I did some googling on how to make category post pages and came across this guide. I followed the guide, replacing the standard WP Loop with the Post Loop from my theme. Actually, I needed a version of my theme’s loop that used $my_query for the post data, so I created a file in my child theme’s directory called loop-cat.php, and placed my very slightly modified code there. I just have my page template (also in my child theme’s dir) include that file.

The page template from that guide include a block of conditional statements at the top, where you have to specify which page ID should load which category ID:

[note color=”#DDD”] [/note]

I’m not a fan of hard coded IDs and such, so I needed to find a better way. A little more googling and I found a way to have the category name determined from the page name. With this setup, you just have to have a category with an identical name to your page and your done. There’s never a need to go in and fiddle with conditionals in your page template. BOOYA!

[note color=”#DDD”] [/note]

 
There was one last thing nagging me about this theme. On the main index page, the Post Info didn’t include sharing links, but on my category pages, search pages, or archive pages it did. I really don’t see why they shouldn’t always be visible, especially since this theme does such a nice job of moving that info out to the left of the post. I tracked down the bit of code behind this behavior to a function in functions.php. I copied that function out to a new file and saved that in my child theme’s directory as functions.php. I changed the IF statement that hid the sharing links on the main page to allow them everywhere and rejoiced.

EDIT: I found out later why the theme’s author had removed sharing links from certain types of pages. It’s because on pages where you see excerpts, like archives, the Post Info boxes will end up overlapping if the sharing links are visible.

Overlapping Post Info

Overlapping Post Info

Ok, so now I understand why it was hidden on some pages, but I still don’t like that solution. If some person out there want’s to share the fruits of my labor, THEN DAMNIT THEY NEED THOSE LINKS!!! My solution was to incorporate a little jQuery into the code to allow hiding and showing the sharing links when “share this post” is clicked on. Now I always have “share this post” visible, but the links aren’t always expanded. A nice compromise if I do say so. Now the archive pages are pretty and shareable.   :)

Unfortunately, they were still overlaping once expanded (on excerpt pages), but I managed to fix that by having the post container expand and contract with the sharing links.

NOT Overlapping Post Info

NOT Overlapping Post Info

What I’ve left out of this telling is the myriad syntax errors, accidental deletions and other shenanigans that are a hallmark of my *cough* method.

Posted in Web Development Tagged ,

Missing Admin Bar in WordPress

I just installed WordPress and after installing a theme I liked I noticed that it was missing the Admin Bar on the public portion of the site.

Missing Admin Bar

Missing Admin Bar

 

A quick google search turned up the answer in the WordPress Support Forums:

[note color=”#DDD”][quote style=”1″]Most likely wp_footer is missing from footer.php. If it is, try adding <?php wp_footer();> to that file just before <?body></html>. [/quote][/note]

That totally worked, but I wonder why the theme wasn’t including the footer?

Site with Admin Bar

Site with Admin Bar

Posted in Googling for Answers, Web Development Tagged

Compress/Uncompress a Linux Directory

I’m so tired of googling this. The reason I have to look it up every-damn-time is because I can never remember exactly what the switches are for compressing/uncompressing.

The following was taken from here.

Create Archive:

[note color=”#DDD”]tar -zcvf example.tar.gz /home/example[/note]

Where:

  • -z: Compress archive using gzip program
  • -c: Create archive
  • -v: Verbose i.e display progress while creating archive
  • -f: Archive File name

Extract Files:

[note color=”#DDD”]tar -zxvf example.tar.gz[/note]

Where:

  • I’m guessing that -x means extract

Extract to a specific directory:

[note color=”#DDD”]tar -zxvf example.tar.gz -C /tmp[/note]

Posted in Googling for Answers, Linux Tagged , , ,

Setting up a LEMP server on Debian 6

>>Skip the long winded description

Downloading Packages
Vhost Configuration
Testing
Installing MySQL
Installing phpMyAdmin
Setting up SFTP

I’ve been playing with linux as a hobby for about 13 years and I’ve been running my own virtual servers for web and mail for about 2. In all that time, my only web server experience has been with Apache. Whether it was at work, or with my personal sites (shared hosting and VM), or my hobby box at home. I recently decided to try using Nginx instead of Apache. I’ve always wanted to chronicle one of my adventures in server land, and this is my first attempt at doing it.

I currently have my virtual machines with Linode, and I have nothing but good things to say about them. The idea with an unmanaged VM is that they don’t have to help you with your shit; you’re on your own. However, they have been very helpful the couple of times I’ve had a question. They also have an awesome library of helpful docs, and a great forum community. For this adventure, I started out by following one of their LEMP guides in the library: LEMP Server on Debian 6 (Squeeze)

Round 1:

After running through the guide, I had a working LEMP server. I run a few WordPress sites, so the next thing I did was install WordPress. It worked pretty nicely right out of the box, but I did have to fiddle with the site’s vhost config to get anything but the default permalinks to work. I’m not going to post that code here, because I would end up finding better WordPress code later at the Nginx wiki.

At this point I got to thinking about some of the other useful stuff that I used to do with .htaccess, and started looking for ways to accomplish the same goals with Nginx. The first thing that came to mind was setting per-directory/site php settings. A bit of googling turned up an answer; put something like this in your vhost config:

[note color=”#DDD”] [/note]

Unfortunately, I was unable to make this work. I tried and tried and tried, using various sytnax changes and config tweaks, but it was a no go. Perhaps the specific versions and methods I had used just didn’t support what I was trying to do, but I don’t really know. I learned a few things while googling though: I learned that there is more than one way to setup Nginx and PHP. In fact, the preferred method seems to be PHP-FPM, which the Linode guide did not use. Here’s a quote from a config guide that is linked to from the Nginx Wiki:

[note color=”#DDD”][quote style=”1″]If you are not using PHP-FPM, you really should be as opposed to the old spawn-fcgi or php-cgi methods. For debian lenny/squeeze dotdeb makes php5-fpm available[/quote][/note]

That touches on another thing I learned: that you can get more up-to-date LAMP/LEMP components from dotdeb.org by adding their repository to your sources.list. Many of the guides I came across did this, so I figured I’d give it a go in round 2.

Round 2

Downloading Packages

I started again with a fresh install of Debian 6. After basic config (setting timezone, hostname, etc) and security setup, I added the dotdeb.org repositories to my sources.list:

[note color=”#DDD”]nano /etc/apt/sources.list
[/note]

Then I grabbed the GnuPG key, added it to the server, and updated APT:

[note color=”#DDD”]wget http://www.dotdeb.org/dotdeb.gpg
cat dotdeb.gpg | sudo apt-key add –
apt-get update[/note]

Next I installed Nginx and PHP-FPM (and some other php modules):

[note color=”#DDD”]apt-get install nginx
apt-get install php5 php5-fpm php-pear php5-common php5-mcrypt php5-mysql php5-cli php5-gd php5-curl[/note]

Remember the quote above about how I should have been using PHP-FPM? Well, that same author had something to say about using unix sockets vs TCP, and I’ve got a good feeling about that guy, so I decided to go the same route.

[note color=”#DDD”][quote style=”1″]I prefer to use a unix socket, it cuts out the TCP overhead, and increases security since file-based permissions are stronger.[/quote][/note]

Speaking of file-based permissions, another thing I came across while googling was how to set up Nginx for a shared hosting environment. I learned that you can configure seperate “pools” for PHP-FPM that will allow each site to execute PHP scripts as the user/group defined in that pool. This should also work well with using SFTP to grant access to a website directory, which I intend to do.

So far, there are 3 major differences between my new setup and the Linode guide:

  1. PHP-FPM instead of PHP with FastCGI
  2. PHP listening on UNIX sockets instead of TCP ports
  3. Defining PHP-FPM pools for my sites to use.

Vhost Configuration

Ok, next up are configuring a virtual host and pool for my site. I’m going to start with the pool, since the settings I use there will influence my vhost config.

The biggest differences between my new pool and default one (www.conf) is that I’ll be using a new user/group that I’ll create, and I’ll be using unix sockets. For a detailed explanation of each line, have a look at the default pool (/etc/php5/fpm/pool.d/www.conf). Here is my example config:

[note color=”#DDD”] nano /etc/php5/fpm/pool.d/example.com.conf
[/note]
Here’s a handy dandy find and replace method for using something like I posted above as a template. Add a little bash scripting to copy the template and do the find/replace, then do that for a few other config files, and you’ve got a pretty simple method of adding a new site to your server. :)
[note color=”#99CCFF”]sed -i 's/searchfor/replacewith/g' /path/to/file[/note]

Next I created the user referenced in that config:

Note: the password chosen here will be used later for SFTP purposes.

[note color=”#DDD”]useradd -M -s /sbin/nologin example
passwd example[/note]

Where:

  • -M: do not create the user’s home directory
  • -s: login shell of the new account

By setting the login shell to /sbin/nologin, I’m effectively blocking login. This isn’t strictly necessary, since this user will be restricted to SFTP only, but it makes me feel warm and fuzzy inside to have another layer of security.

Next I created my vhost config. Take a look at the default config (/etc/nginx/sites-available/default) if you’d like to see a commented example. Also, right up at the top you’ll find this:

[note color=”#DDD”]# You should look at the following URL’s in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
[/note]

I took the comment’s advice and visited those sites, and I’m so glad that I did. If you only check out one of those sites, let it be the Pitfalls. Turns out that the vhost example in the original Linode guide contained a number of those pitfalls. Here are some things I changed thanks to the Pitfalls page:

  1. I moved the root directive from the location block to the server block. (To be fair to the Linode guide, their first example puts it in the location block, but the later example which includes a PHP section has it done correctly.)
  2. I moved the index directive from the location block to the http block (in /etc/nginx/nginx.conf)
  3. I took the hard coded path out of SCRIPT_FILENAME and replaced it with $document_root

Here is my example vhost config:

[note color=”#DDD”]nano /etc/nginx/sites-available/example.com

[/note]

That config is based on one that you can find by following the Configuration link above. It’s under PHP, WordPress and Drupal Configuration Examples. Following the advice on that page, I created two includes: drop.conf and php.conf. The first one, drop.conf, denies access to various things like hidden files, while php.conf contains the basic php config that I’ll use for most sites. You can find commented versions of these files at the aforementioned link. You’ll notice that after I include php.conf I define the unix socket that this vhost should use. I can’t put that line in the conf since it needs to be unique for each vhost.

[note color=”#DDD”] nano /etc/nginx/drop.conf
[/note] [note color=”#DDD”] nano /etc/nginx/php.conf
[/note]

Then I put the index directive in the http block of nginx.conf. I won’t need to put it in a vhost config unless a site has a special need for it.

[note color=”#DDD”]nano /etc/nginx/nginx.conf
[/note]

One more thing. Both the Configuration example I linked to above, and the default vhost config say to set cgi.fix_pathinfo to 0 in php.ini, so I did that.

[note color=”#DDD”]nano /etc/php5/fpm/php.ini

[/note]

And here’s the example’s explanation for why I just did that:

[note color=”#DDD”][quote style=”1″]REMEMBER: Set cgi.fix_pathinfo to 0 in the php.ini when using Nginx with PHP-FPM/FastCGI, otherwise something as simple as /forum/avatars/user2.jpg/index.php could be used to execute an uploaded php script hidden as an image.[/quote][/note]

There is a section in the Pitfalls page that details the issue and lists several ways to address it. I went with “try_files $uri =404;” in the php block of my vhost config. Read the Passing Uncontrolled Requests to PHP section of the article for more information.

Next I created a symbolic link from my vhost config in sites-available to sites-enabled:

[note color=”#DDD”]ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/example.com[/note]

I also removed the symbolic link for the default site:

[note color=”#DDD”]rm /etc/nginx/sites-enabled/default[/note]

Then I created the directories referenced in my vhost config and assigned ownership to the user/group defined in the pool for this site:

[note color=”#DDD”]mkdir -p /var/sites/example.com/public_html
mkdir -p /var/sites/example.com/logs
chown example:example /var/sites/example.com/public_html[/note]

Testing

Finally, I created an index.php file, rebooted nginx and php, then checked to see if everything was working.

[note color=”#DDD”]nano /var/sites/example.com/public_html/index.php
[/note] [note color=”#DDD”]/etc/init.d/nginx restart
/etc/init.d/php5-fpm restart[/note]

Everything worked! I was able to browse to my site and see “OH HAI WORLD”, which made me giggle.

I also created a phpinfo file, so that I could make sure PHP was running as the user specified.

[note color=”#DDD”]nano /var/sites/example.com/public_html/phpinfo.php
[/note]

In the Environment variables I could see that USER reflected the user that I defined in the pool for my site. HOORAY!

One last test was to try and change a PHP ini setting using the method listed at the very beginning of this post. I really held my breath for this one, but it worked!!! Color me happy.   :)

Installing MySQL

Many of my sites are running WordPress, so I still needed to install MySQL and phpMyAdmin, and tweak the vhost config to work better with WordPress. First I installed the MySQL packages. During the installation process you will be prompted to set a password for the MySQL root user.

[note color=”#DDD”]apt-get install mysql-server php5-mysql[/note]

After they installed, I issued the following command to secure MySQL:

[note color=”#DDD”]mysql_secure_installation[/note]

First I had to type in the password that I created in the previous step. I said “no” to changing the root password and “yes” to everything else.

Then I was able to login to MySQL:

[note color=”#DDD”]mysql -u root -p[/note]

I’m not a MySQL guru, so I’ll be using phpMyAdmin for most of my database administration.

Installing phpMyAdmin

First, I installed the package:

[note color=”#DDD”]apt-get install phpmyadmin[/note]

It automatically launched the setup, and here are the settings I chose:

[note color=”#DDD”]Web server to reconfigure automatically: none
Configure database for phpmyadmin with dbconfig-common? No[/note]

The reason I did not select an option on the first question is because the only options were Apache or Lighttpd. As for answering “no” to the second question… I was just following a guide. :) That guide also had the configuration I used for my vhost config, which I only modified slightly.

[note color=”#DDD”]I added this to my site’s vhost config:
[/note]

One caveat: because of the fact that my website is running php as it’s own user, I had to change the group of /var/lib/phpmyadmin/blowfish_secret.inc.php to match the group of my php user. (It was previously set to www-data)

[note color=”#DDD”]chown example /var/lib/phpmyadmin/blowfish_secret.inc.php[/note]

Setting up SFTP

I used this Linode guide as a baseline, but I setup my sshd_config a little differently than their guide. I setup a Match block to restrict the user “example” to SFTP and the web site directory /var/sites/example.com. That directory is owned by root, but has a sub directory of public_html that is owned by example (the user). With this setup, that user can login and see their logs and the public_html directory, but they can only write to public_html.

[note color=”#DDD”]nano /etc/ssh/sshd_config
[/note]

You’ll notice that this user is allowed to authenticate via password. In the main config and I have PasswordAuthentication set to no and I use SSH Keys to access the server, but when granting SFTP access, I want the experience to be as close to traditional FTP as possible.

NEXT: wordpress vhost config, bash script for adding sites

Posted in Googling for Answers, Linux, Nginx Tagged , , ,