Friday, June 26, 2009

Exporting Windows Event Viewer data for compliance

Exporting Windows Event Viewer data for compliance

The companies that are subjected to regulatory compliance are often required to store and archive the logs from various part of their infrastructure such as applications, firewalls, VPN and servers. Most of the network devices support Syslog and if you have any syslog server in your environment you should be able to view, collect and archive the syslog data. Kiwi Syslog server is one of the best tools available in the market.

Windows servers do not have a syslog client by default and usually all the system related warnings, alerts and information are stored and displayed in the Windows Event Viewer. Event viewer allows exporting of data locally in different formats for review. However, in an enterprise environment, there is no tool exists to automate the collection of event viewer from a centralized location.

One great solution for this is using software called ‘winlogd’. Winlogd converts the windows event viewer logs into syslog and send it to the syslog server. Winlogd installs itself as a windows service and requires a registry edit to specify the syslog server IP.
It can be easily pushed to all the servers in an enterprise environment using a .reg file.
Once the syslog server can receive the data from servers, it can be viewed and archived for compliance purposes.

One limitation of Winlogd is it doesn’t allow filtering the window event viewer logs. So, all the data that is going to Windows Event Viewer (including ‘information’) will be sent to syslog server. If you have many chatty servers that would cause lot of informational event logs, it may generate tons of syslog data and network traffic. I’m hoping that winlogd community will fix this in their next release. Nevertheless winlogd is a great tool!

More information on ‘winlogd’ can be found here:

http://edoceo.com/creo/winlogd


Friday, June 19, 2009

Why does IP address and subnet mask matter in an enterprise network?

Why does the IP address and subnet mask matter?

We had an interesting issue that relates to networking and SQL heartbeat with one of our clients that I would like to share.

One of our clients has their servers distributed across 3 data centers in US. Let us call them A, B and C. They have a VPN tunnel connecting all their sites together using Cisco ASA firewalls. Their IT staff access the entire infrastructure through one data center (A) which has remote access VPN enabled in the Cisco ASA. They recently reported an issue of not being able to access the servers in SQL cluster across remote access VPN located in data center (B). However, those servers are accessible from data center A, C and within B itself and others servers in data center B were accessible through remote access VPN.

As their remote VPN connection terminates at Cisco ASA at data center (A), various teams were involved to find out the cause. The following were checked to identify the cause of this issue:

It was ensured that the remote access VPN subnet (10.1.1.x) is added to the crypto-map on the site to site VPN configuration.
We also made sure all the servers are connected to the same switch and residing on same VLAN.
There was no specific access list (ACL) or firewall or IPS or F5 configuration that was blocking traffic to the database servers from remote access VPN subnet (10.1.1.x).
We ensured all the servers have the same IP default gateway configured.

During packet tracing, it is found that the traffic reaches to the ASA at data center A and also reaches data center B. It is found that the traffic was not going back from the servers to the remote access VPN subnet. We realized that there would be something wrong on the SQL cluster servers and started looking in depth on its network configuration. We identified that SQL server’s heart beat NIC was configured with the IP address of 10.0.0.x with a subnet mask of 255.0.0.0 (/8) allowing to have 16777214 hosts where only 2 IP addresses are needed for heart beat. So, all the incoming traffic from remote access VPN was forwarded to the heart beat NIC on the SQL servers and not going back to the remote access VPN ASA.

Having a subnet mask of 255.255.255.252 on SQL servers heart beat network would have allowed it to have only two IP addresses that are needed for heartbeat on the SQL cluster. As it is a production SQL network, we did not want to change the heart beat network’s IP address or subnet mask. As an alternative workaround, we used persistent route in Windows 2003 to configure the remote access VPN traffic to reach the correct NIC and gateway. A helpful article on windows 2003 routing can be found here. Once we added the persistent route, the remote access VPN users were able to access the SQL servers.

Lesson’s learned:

Make sure you aware of all the network addresses and subnets involved in all the locations.
Assign a subnet mask for the required number of host addresses.