Announcement

**Midorikawa** · 12-05-2010, 11:05 PM

Well, as a linux systems administrator, I believe I'm qualified to throw my 2 cents in. :-)

1) FreeBSD is REALLY secure, pretty much out of the box. Just get pf working right, and you're golden. FBSD has a great track record security wise. That said...

2) Gentoo is basically FBSD with a linux kernel, which I'm sure you're aware of. Hardened Gentoo is one that I've been playing with recently as of late, and while I haven't gotten the kernel booting right just yet, it's looking promising. My install will use SELinux, but be aware, that SELinux WILL NOT come fresh from /usr/portage/distfiles preconfigged. Be prepared to spend plenty of time going through and getting everything working nicely. Just like a firewall, SELinux will require lots of time and love.

3) Tripwire looks to be abandonware. I'd be wary of using such software in a production environment. If your company has a dev team, I'd much rather have them develop something that watches for changes in config files, and if found, overwrite them. Yes, it won't tell you if someone changed something, but in house is better (and more secure!) than abandonware.

4) Bastille looks to be in the same sad state of affairs as tripwire, not having seen an update since 2006. Active development is a good thing for someone who's security conscious.

ClamAV will help reduce the spread of malware from client to client (assuming windows client -> linux server config), but remember to keep it updated. Gentoo's default install is a version old, older if you're not using ~ARCH.

5) I know everyone tells you not to use ~ARCH in your keywords, but unless you have something weird or non stock running on your servers, ~ARCH isn't necessarily bad. I've found through years of Gentoo use (started with Gentoo in 2004 :-)) that ~ARCH is necessary for latest version releases, which often include security updates rolled into feature updates, or other more minor updates that wouldn't otherwise be in ARCH. This is where a test environment becomes a MUST HAVE, BEFORE you roll anything to production.

6) Be aware that Gentoo is NOT a set it and forget it OS. It requires lots of love and care. Granted, this love and care is something any sysadmin should be doing anyways.

7) Finally, MAKE SURE you have portage-utils, and gentoolkit installed on your system. Another good one to use is demerge. It lets you take a snapshot of your system, and roll back if you have issues. This is far easier than "well...wtf updated to break EVERYTHING on my server?!" Gentoolkit includes the glsa-check utility that scans your portage database for vulnerable software, for example:

(root@domain[~]# glsa-check -t all 9:58:23)
This system is not affected by any of the listed GLSAs

I'd have your emerge --sync run nightly, followed by an emerge -pvu world that gets emailed to you (and other important staff of course!) nightly. I'd also have a glsa-check run daily that also gets emailed to you. those 3 alone should go a long way.

PS, make sure you log to an external log host with syslog-ng. This can prevent some log munging by hackers, and can reveal valuable insight into what's going on at any given time, no matter who's in the system. :-)

Sorry for the huge novel, but I figured it's best to inundate you with info than throw a drop or two your way at a time. :-P

**pitchpole** · 12-06-2010, 12:52 AM

Midorikawa,
Thanks a bunch! I hadn't noticed that tripwire and Bastille were now abandonware. I remembered them from the first server I set up way back in '06 or '07 (and haven't had occasion to build a public-facing server since then). And hadn't thought to check the dates of last update >.<

That said, we're a php shop. Unfortunately developing our own security software isn't going to happen. We have three people (including me) who know c/c++/something-not-php. Unfortunately we're all busy with other projects. Heck, I had to leave my project to deal with this. So we would need to rely on something already available. Now I'm cribbing off of wikipedia, but what about OSSEC (http://www.ossec.net/) and/or AIDE (http://aide.sourceforge.net/), both of which look updated (I checked this time!)?

I also hadn't realized that FreeBSD had such a good record. Mostly I was interested in Hardened Gentoo because of PAX/PIE and the builtin SELinux. In theory it ought to help against unknown vulnerabilities. My secondary motivation for using Gentoo is having absolutely no software that I a) dont know about and b) don't need on the server. I really like building the system up from scratch. I've mostly used Gentoo on really low power machines (Think better than a decade old). I've never actually built a FreeBSD system, so I don't know how basic the system is when built.

As to ClamAV - I'll probably drop it then. Some of the documentation looks as if it is claiming that it can find linux/unix malware (which was the idea of having it - finding anything that makes it on). However between the other stuff that will tell me what changes and the tool I wrote this morning (python ftw!) which iterates over the file system to look for changes, I'm probably pretty well set on finding the changes. And given that I plan on keeping an image I can roll back to, I'm probably well set on getting malware off with a minimum of damage and time.

Thank you very much for all your advice with respect to running Gentoo. I definitely picked up some new tricks. Having only run it as a desktop environment, I did have worries about how to deal with it as a server. I think you eliminated all of them.

All told, thank you very much for your advice! I can see how this will work, but its looking increasingly more likely that I'll have to come up with a short term solution and then build a better system to replace it.

Thank you very much,
pitchpole

**Pedersen** · 12-06-2010, 12:58 AM

You are going to see me suggest something that will have people screaming at me, but it's a simple truth: Windows Server, administered by a competent admin, is more secure than OpenBSD administered by a fool.

The correct operating system to use, for security, is the operating system that you are able to secure, plain and simple.

If you are able to secure FreeBSD, then that is what you should use. If it's Windows, Gentoo, Redhat, whatever you can secure, that is what you should use.

My personal choice, on the server, is to use Debian. The first reaction to that suggestion is to point out how old Debian packages are. That's right, they are. They also have security fixes backported to those versions, and go to pretty much unheard of lengths testing their upgrade processes. If you stick with their stable distribution, you're going to be very unlikely to have issues with them.

My home server has a pretty bad uptime right now. It's only at 123 days. What can I say, I had a power outage a few months ago. Your mileage may vary.

**Midorikawa** · 12-06-2010, 04:32 AM

Quoth pitchpole View Post

That said, we're a php shop.

This alone tells me you REALLY need a system YOU know how to secure. I can secure a gentoo install in my sleep, because I know the ins and outs of linux, plus the gentoo way of doing things. That doesn't mean that that's the direction you should go. I'm sure you're aware, but PHP isn't the most secure system in the world. It can be made secure, but default builds get newbies hacked thousands of times a day, if not more.

Quoth pitchpole View Post

Now I'm cribbing off of wikipedia, but what about OSSEC (http://www.ossec.net/) and/or AIDE (http://aide.sourceforge.net/), both of which look updated (I checked this time!)?

Aide can work, but again, you REALLY need to know what you're doing. I don't remember where I heard this before, but a poorly configured firewall (iptables) is actually MORE of a security risk than none at all, and it's easy to see why. If you don't have it configured in a sane manner, all you're doing is telling hackers "hey, I don't know how to do it right, so I'm faking it as I go." This 1) turns you into a target, 2) lets them see more about your system than you'd really see otherwise, due to simple things like rejecting, not dropping, bad traffic, or not properly handling malformed packets. Since iptables runs at the kernel level, that's a bad thing.

Quoth pitchpole View Post

I can see how this will work, but its looking increasingly more likely that I'll have to come up with a short term solution and then build a better system to replace it.

Full stop. stop-gap solutions are a bad idea, and lead to "oh, I'll get to it later."-itis. do it right the first time, and do it ONLY once you have a full understanding of everything involved in the undertaking, or find another way to do it.

I don't mean to sound rude, but you seemed a bit more knowledgeable in the first post, just a bit out of date with your knowledge. Desktop and Server environments are COMPLETELY different, and trying to apply your knowledge of desktop environments to a server is a good way to get your job taken from you by someone in Hungary.

As Pedersen said, you REALLY need to know what you're doing before you do this. If you don't know how to build a good Gentoo server, but you can do so with Debian, FreeBSD, Windows, or hell, even OS X Server, do it that way.

I understand the appeal of Gentoo. All my servers have been Gentoo for probably about 5 years. Just don't get in over your head with your job, especially with the job market the way it is.

**pitchpole** · 12-06-2010, 05:18 PM

Both,
Thank you very much for your advice. The timescale for me building up a more secure system increased a whole lot - it turns out the upstream network screwed up and our server was never infected. So I have the go ahead and build a more secure operating system on my timescale.

Pedersen,
First off, apologies that you couldn't see my earlier reply - I'm still a moderated poster.

I completely agree with you. Hopefully I'm not the fool with OpenBSD. Or whatever system I end up using.
There are two big reasons that we're going with a Linux/Unix operating system. First, my boss really likes open source software. Second, I haven't used Windows since XP, so I'm not at all familiar with it. And as your above statement, if I can't secure it, it isn't the system for me.

As to debian - I can certainly see why you would like it. I use it a lot in virtual machines, I've paid for hosting on it. I like it - its a wonderful system. The next personal machine I build will probably be running a debian variant. But I don't know if its the best suited for our particular purposes.

As to uptime.. Our production server is at 180 days, development at 90-ish. Both are longer than I've been in the shop. In general, I feel as though any reasonably mature linux/unix system will probably run into more uptime issues from external factors (kernel updates, power outages) than from software instabilities.

Midorikawa,

First off, an apology. I didn't give you all the context of the situation. I imagine I gave you the impression I'm a professional in a professional shop. Truth is, I'm a student at a student job. As many other students on these boards have pointed out, semesters everywhere are coming to an end. Personally, this is the last day I work before mid-january, when I get back after break. That is the only reason I considered a stopgap. I hate the idea as much as you do, but my shop needs this particular server. It doesn't excuse it, but I hope you can at least understand the why. That said, I now won't have to, as it turns out the upstream network got the wrong server.
I also managed to show all the areas in which I'm inexperienced. I don't take offense that you think I sounded moronic or incompetent in my second post - I agree with you. My first post is my general level of knowledge - give me the time and I will get you a system as perfect as I can make it. I'll take the time to learn what I need to, to ask others more knowledgeable than I am, test and tune. My second post shows my inexperience with this situation. I'm not terribly used to deadlines or having to make things happen quickly.

Okay, on to actual response: The reason I'm looking at gentoo is that other than drop-n-go install for workstations, all my significant work in the last couple years has been in gentoo. My boss has given me permission to build this new system my way, which means I can take the time and get it right. It one of the strong reasons I want to go with gentoo (beyond the "it's what I'm most familiar with") - since I'm making so many specific choices about what goes on the system, I end up with a much better understanding of what is going on.

I feel as though I might be defending myself. Hopefully I don't come across that way - I really appreciate your advice. I'm doing my absolute best to make this system the best I can. I'm following the advice of both of you (and of others) - I'm sticking with what I know best (gentoo) and taking the time to learn the things I don't know.

Thank you very much for your advice,
pitchpole

**Midorikawa** · 12-06-2010, 08:51 PM

Quoth pitchpole View Post

I also managed to show all the areas in which I'm inexperienced. I don't take offense that you think I sounded moronic or incompetent in my second post - I agree with you.

You sounded neither moronic nor incompetent. I just wanted to make sure you REALLY understood what you were getting into before you dove into a really complex project that won't be easy or simple to setup. In addition, it's easier to have a more commonly known system for future support by other admins...unless...you really want to guarantee job security. :-P

You, above all, sound inexperienced. Combined with Gentoo in a production server environment...that can be dangerous.

Hell, I have 6 years of Gentoo under my belt, and I STILL manage to royally screw things up every now and again. Last year, I killed off /usr while trying to move it to a new drive in a live system. Yes, I should have downed the server and done the move offline, but I managed to recover, and I did so with 0 downtime for any services. :-)

**pitchpole** · 12-07-2010, 12:16 AM

I understand what I'm getting into. I know this is a really complex project that I'm going to need to spend a lot of time on. No matter what system I end up with, it is going to be a complex project I spend a lot of time on. So I should do it with the system I know best.

I agree with you about the future support by other admins. That exact point is one I'm upset about - the previous admin left next-to-no documentation. He did a verbal braindump towards two people and just assumed they would remember everything. Add to that my boss's "document everything" kick, and one of the reasons to do this is to build up a sysadmin manual for us. Why every choice was made, what it effects, how to do simple tasks, how to do more complex tasks, what is where, what to watch for, what to do when item X happens. Hopefully building up an ops manual ought to be enough to let me be replaced fairly easily.
I'm aware there are going to be screwups. And honestly, I don't know what to say other than I'll deal with them and I hope future admins will as well.
And I must admit to being very impressed you managed that.

Thank you for all your advice - I really do have a better understanding of what I'm getting into. I've very much upped the amount of time I expect to spend on this, but I still think I can get the results my shop needs.

pitchpole

Announcement

Unix/Linux security Recommendations

Unix/Linux security Recommendations

Comment

Comment

Comment

Comment

Comment

Comment

Comment