Trend Deep Security Manager 9 – Install and Configure (again!)

While working on a recent project for a client utilising Cisco UCS and NetApp for a cloud offering I was tasked with getting Trend Deep Security 9 working for a multi-tenant cloud environment. The primary caveat is that the environment isn’t true end-to-end multi-tenancy as the virtualisation layer is not fully segregated. vCloud Director or another similar tool has not been used but rather the vCloud Suite from VMware and segregation is at the network and storage layers through the use of vDCs on Nexus 7k (network) and SVMs on NetApp Clustered Data OnTap (storage virtual machines). In the production environment Trend Micro professional services were engaged to deliver the original design. Part of the criteria given to them was not to enable multi-tenant mode within the Shared Resources cluster as the tenants would not be managing their own anti-virus protection or scanning. In order to satisfy the requirements of multiple VMware clusters protected by one Anti-virus package a DSM was deployed on each cluster and managed from a central console within the Management cluster. I will go into more of a discussion on the ideal architectural design for multi-tenant anti-virus in another posting.

And so to the beginning of the troubles. No anti-virus solution is really ever straight-forward. There’s a number of policies and exclusions to consider for both operating systems and specific applications and usually there is lack of distinct information within the installation and admin guides. Trend Micro Deep Security Manager is no different. This however is not a huge criticism of Trend, they have to make their documentation as generic as possible for multiple use cases. It does make installation just that bit more frustrating though. You can find the full Deep Security 9.0 Installation Guide here.

Trend Micro Deep Security consists of a number of components that work together to provide protection against viruses and malware in real-time. It can also provide Intrusion Prevention, Web Reputation, Firewalling, File Integrity Monitoring and Log Inspection. It is also available as both agent-based and agentless options. The component of Trend Deep Security are:

• Deep Security Management Console (DSM) – this server (recommended to be virtualised) is the central web-based management console for controlling and managing all Deep Security enforcement components (DSA’s and DSVA’s). The Server is recommended to be Windows Server 2008/2012 R2 64bit.. It is important that it is installed on a different ESXi host to that hosting the VM’s which are protected by the DSM. The DSM should be allocated 8GB and have 4 vCPU allocated. This configuration will be capable of serving up to 10000 agents. The MS SQL database size is relatively small at around 20GB for 10,000 agents.
• Deep Security Relay (DSR) – this server is responsible for contacting Trend Micro’s Security Centre for collection of platform and security updates and relaying this consolidated information back to the DSM and to Agents and Virtual Appliances. The DSR will also be virtualised at Interactive with 8GB ram and 4 vCPU. This configuration will be capable of serving up to 10,000 agents. The Relay has an embedded Agent to provide local protection on the host machine. In the case of multiple relays each will act independently and synchronise their local databases with the Trend Security Centre.
• Deep Security Virtual Appliance (DSVA) – this server is a virtual machine appliance that is installed on every ESXi host. The DSVA enables agentless Deep Security control and management within the hypervisor, providing Anti-Malware, Intrusion Prevention, Integrity Monitoring, Firewall, Web Application Protection and Application Control protection to each VM. The agentless control is only currently available for vSphere 5.1 or earlier. Support for VMware 5.5 will be available in 2014 Q2. The DSVA will communicate directly with the DSM and it is recommended to enable affinity rules within VMware to lock each DSVA to their required ESXi host.
• Deep Security Agent – for non-Windows servers (such as Linux), the agent is deployed directly to the VM’s OS computer, providing Intrusion Prevention, Firewall, Web Application Protection, Application Control, Integrity Monitoring and Log Inspection protection. This is the traditional client-server deployment model and the agent could be included within the imaging process or pushed out from the DSM. DSA will also be necessary on all VM’s within a vSphere 5.5 hypervisor until Q2 2014 (after which a DSVA can be used with vSphere 5.5).
• Smart Protection Server. Web reputation works by clients contacting Trend Micro’s Smart Protection Service on the Internet. Rather than all clients accessing this service, it is possible to deploy Trend Micro’s Smart Protection Server as a VM. The Smart Protection Server will periodically update its URL list allowing it to locally respond to client requests for web reputation ratings. This component is normally part of the Trend Micro Office Scan products and using it may incur an additional licensing fee. Given that the DSVA also caches similar data, this product is not recommended. Hence, the DSVA and DSA will regularly be checking web reputation over the Internet.
• Deep Security Notifier – is aWindows System Tray application that communicates the state of the Deep Security Agent and Deep Security Relay on local computers. A DSA and DSR already contain the Notifier but for Windows guests protected by the DSVA will need ti install the Notifier as a standalone application.

Deep Security Feature Modules

• Anti-Malware
• Web Reputation
• Integrity Monitoring (tracks both authorised and unauthorised changes to files and system registry in real-time).
• Intrusion Prevention (has out-of-box vulnerability for over 100 applications and helps to shield known vulnerabilities and zero-day attacks from unlimited exploits until protection can be patched. For example, it can help to protect web applications vulnerabilities such as SQL injection attacks as well as identifying potentially malicious software accessing the network.)
• Firewall/Deep Packet Inspection (centralised management of server firewall policies using a bi-directional stateful inspection firewall and deep packet inspection).
• Log Inspection (identifies relevant security events that might be buried in multiple log files.)

Below are the steps I followed to implement Trend in our test environment. Some things that are worth considering in advance of deployment if not already specified in a design are:

• IP addresses of each DSM and the DSVAs
• If crossing VLANs ensure the required ports are opened on any firewalls (in my case everything was on the same management VLAN and was routed on Layer 2 through the Nexus 5k sitting just above the virtualization layer)
• User accounts for backend database, for vCenter access and also LDAP synchronization

Port Requirements:

DSM <-> DSVA

Port 4118, 4122

DSVA -> DSM

Port 4119, 4120

ESXi Hosts -> DSM

Port 4119

DSM/Relay -> Database server

Port 1433

DSM <-> LDAP (domain controller)

Port 636 (Secure LDAP)

 

Step 1 – Deploy a Linux VM for Trend DSM:

• Deploy a new Linux VM from template. I am using RHEL 6.2 (Santiago). Edit the vNIC to be on the required VLAN.
• Edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file for the hostname and the IP address of the DSM
• Edit the /etc/resolv.conf file to point to the correct DNS servers and searchdomain is set to the domain name rather than localhost.
• Run the command: service network restart

Once the VM reboots I tried to ping it but as unable to get a response. In order to fix this issue I disabled IPtables using the below command and ran the ping again which worked:

• service iptables stop
• chkconfig iptables off
• service iptables status

Step 2 –  Create a database on SQL Server

The database requirements for multi-tenant mode is very different to normal mode so before proceeding with the database configuration speak to your architect or Trend professional services engineer. In my case I’m not using multi-tenant mode as the clients will not be managing their environment themselves.

To create a new database right-click on databases at the top of the tree and select New Database. Enter Trend as the database name. Leave the default settings as they are and deploy the database. Once the database appears you can then create the SQL user account.

Step 3 – Create user account for database and assign the correct privileges

A SQL account needs to be created with the required permissions to effectively use the database created in the next step. When creating a SQL Server database, the SQL Server account must be granted DB_Creator server roles and DB_Owner of the Deep Security Manager Database. I created an account called svc-TrendDSM. Once the server roles were set to allow db_creator as per:

I then had to change the User Mapping for the account to the Trend database using the dbo schema

The last thing to check is the explicit permissions for the user account. As mentioned earlier the permission Connect SQL will need to be granted on the account. Click on Securables and scroll down the Explicit permissions to view if Connect SQL has been enabled.

Step 4 – Create domain based service account

Log onto your Domain Controller as a user with the required permissions to edit user accounts. Within the Service Accounts OU create a new domain account called svc-trendmicro-dsm (or similarly named account) and assign it a secure password. Once the account has been created you can give it permissions to various groups. In my instance I added the account to the admins group for vCenter Server so Trend can synchronize with vCenter execute commands on the ESXi hosts if required.

Step 5 – vShield Endpoint Installation

In order to be able to install vShield Endpoint vShield Manager needs to be deployed in your environment. This can be deployed as a virtual appliance and can be licensed for usage from vCloud Suite licensing or as a purchased stand-alone license.  I won’t go into the details of applying licensing. I also won’t go into the details of installing and configuring vShield Manager as there’s loads of information available on the Internet around the installation. Once vShield Manager (VSM) has been installed then the vShield Endpoint driver needs to be installed on the ESXi host. Open the web console for VSM and log in using an administrator level account, most likely the default admin account should work. Open up the Datacenters and then the cluster where the ESXi host resides. Under Summary you will see 3 options, vShield App, vShield Endpoint and vShield Data Security. Any component that is not installed will show Install as the option otherwise it will show uninstall. Initially when you load this summary vShield Endpoint is set to Install at which point you click Install, click OK on next screen and then the driver will be installed on the ESXi host.

 

If there is a failure with the installation you will be notified at this point. Don’t proceed with the Trend installation until all the issues have been resolved. During my own installation I had issues with vShield Manager not reporting correctly on the status of Endpoint on the hosts. The error I got – Not applicable to ESX version below 4.1 Patch 3 is discussed in another posting.

Once Endpoint has been installed correctly when you view the Endpoint tab you will see the VMs on the host protected and have Thin Agent Enabled in their description.

 

The last thing to check to confirm the installation has been configured properly is to check the Networking settings on the ESXi host to ensure that a new Standard vSwitch was created. This will be an internal vSwitch with no adapters. The filter drivers on the ESX host will redirect all traffic through the internal switch. If this vswitch is not configured then the installation of Trend Deep Security will fail. Go to the ESX host, click Configuration and select Networking and ensure the vmservice-vswitch was created with the vmservice-vshield-pg and the vmservice-vmknic-pg port groups.

Step 6 – vShield Endpoint VMware tools configuration

Once VSM has been configured to install vShield Endpoint on the ESXi hosts the next step is to install the VMCI driver. This is embedded in VMware Tools as the vShield Endpoint driver but is disabled by default and needs to be configured. If it is not already installed as part of the OS image then go to Add/Remove Programs in Control Panel and modify the features installed in VMware Tools. Select vShield Endpoint driver from the list of features and install on local hard-drive. Once the modification has been completed you can run msinfo32.exe from the run dialog box. This will launch System Information window. Select Software Environment -> System Drivers and verify that vmci.sys and vsepflt.sys drivers are running. Once they are running then realtime malware protection will be available. This is only available in Windows Servers. For Unix/Linux you will need to install the agent software from Trend.

Step 7 – Download the required software:

Go to http://downloadcenter.trendmicro.com/ and download the software for Deep Security Manager, Deep Security Virtual Appliance, Deep Security Agent and Deep Security Relay.

The software for version 9 Patch 1 can be accessed via the below links:

• Deep Security Manager v9 Patch 1
• Deep Security Virtual Appliance v9 Patch 1
• Deep Security Agent & Relay v9 Patch 1

Once downloaded you need to copy the files for the DSM up to newly deployed server. WinSCP can be used to upload the file.

SSH to the server via Putty and go to the folder where the files have been placed. Change the permissions of the files so you can run them:

Chmod 777 Manager-Linux-9.0.5370-x64.sh

Step 8 – Run DSM Installation

Before you begin the installation of the DSM you need to specify some information in a dsm.properties file located in the same folder as the .sh installation files. If you run the installation file without having the dsm.properties file specified you’ll run into Java based errors. Create a text document with information similar to below and copy to the DSM using WinSCP.

Contents of dsm.properties

---

LicenseScreen.License.-1=

CredentialsScreen.Administrator.Username=<AdminUserName>

CredentialsScreen.Administrator.Password=<AdminPassword>

DatabaseScreen.DatabaseType=Microsoft SQL Server

DatabaseScreen.Hostname=<SQLServer Hostname or IP Address>

DatabaseScreen.DatabaseName=<Trend Database created earlier>

DatabaseScreen.Transport=TCP

DatabaseScreen.Username=<Trend DB account configured earlier>

DatabaseScreen.Password=<Trend DB password configured earlier>

DatabaseScreen.SQLServer.Instance=

AddressAndPortsScreen.ManagerAddress=<FQDN of DSM server> (Do not use IP address as this will cause problems for some other server connections)

AddressAndPortsScreen.ManagerPort=4119

AddressAndPortsScreen.HeartbeatPort=4120

AddressAndPortsScreen.NewNode=True

CredentialsScreen.UseStrongPasswords=False

SecurityUpdateScreen.UpdateComponents=True

SecurityUpdateScreen.UpdateSoftware=True

---

Once the dsm.properties file has been created you can run the following command to kick off the installation:

./Manager-Linux-9.0.6019.x64.sh -q console -varfile dsm.properties

Once the installation begins you will see files, directories and security policies being created

Once the DSM installation has completed you will be able to access the web console via https://IPaddressofDSM:4119 using the administrator credentials specified in the dsm.properties file. Ensure as part of this that IPtables has been disabled. You can review the steps above on IPtables to confirm they have been disabled. When you log into the web console for the first time you should see something quite similar to below.

Now that the DSM has been deployed, there are some administrative tasks that need to take place. These include the DSM integration with vCenter and vShield Manager as well as setting up the relay so updates can be downloaded.

Step 9 – Connect DSM to vCenter and VSM

Now that the DSM has been installed and you can log into the web console the next steps to be carried out involve integrating the DSM platform with both vCenter and vShield Manager. Go to Computers and click on New and select Add VMware vCenter.

Next you will need to enter the IP address or the FQDN of the vCenter server and use the domain service account created earlier that has the required access to vCenter

You will also be asked to connect to vShield Manager. You can connect using a specific account or just use the admin account. To use an AD account you will need to have SSO configured within vShield Manager.

As part of this integration with vCenter and vShield Manager you will need to access the built-in SSL certificates. Once that has been done you will need to wait just a few moments while DSM connects to vCenter. Once the connection has been made you will see the number of devices and hosts that will be included as part of vCenter

When you click Finish it will take a moment for all the devices to be added and then you reach the screen which advises on how to Prepare the ESXi hosts.

Step 10 – Configure policies, exclusions and scanning configurations

One of the key areas within Trend that doesn’t get much focus but it at the core of what makes it work and operate as you would like are the policies, lists and scan configurations. Before building the policies it is recommended to first work on the directory and process exclusion lists. Click on Lists on the left-hand menu. There are a number of lists pre-configured for us and can be utilised to create different lists for different platforms. The lists include Direct, File Extension, File lists, IP lists, MAC lists and Port lists. To create a new list select the list type, i.e Directory Lists and select New Directory List

Give the list a name and in the Directory(s) section enter the directory listing that you would like to exclude. As this list is used within a scan configuration and later a policy the Assigned To list will be auto-completed.

Generally you won’t need to change the IP lists, MAC lists or Port lists. Please refer to Trend recommendations for what the directory lists, file extension lists and file lists should be set to. One of the policies that needs to be implemented is the Process Image Files which marks the processes that need to be ignored as part of the scanning configurations. Systems such as Citrix etc should be added here.

The next steps is to create the Scan Configurations. These should be based on the types of system files and directories that need to be excluded from scans so the the policies that use the scan configurations don’t overlap. This really just makes it easier for management as the Policies applied to protected devices can only have one assigned policy and that should be based on the primary application installed on the server such as SQL, Exchange etc. There are a number of default scans which can be duplicated to help you get the ball rolling. For my environment I crated scans for General Windows Servers, Exchange Server, SQL servers and RHEL servers. There are 3 different scan configurations for each, a scheduled scan and manual scan have a Scan Type of Manual/Scheduled and the Real-Time configuration has a Real-Time scan type. Real-time scan uses IntelliScan as part of its file type scanning whereas a manual or scheduled scans all files.

You can add a new scan configuration or if you duplicate an existing one you can just edit a configuration. In either case the the things that will need to be configured are the Files to Scan, the different Scan Exclusions based on the lists created earlier, what the actions should be when malware is detected and finally some options to ebable better performance. Below are a number of screenshots from one of the policies. Modify the exclusion lists and scan settings as required.

Now that the scan configurations are in place as well as the directory exclusion/scan lists we can begin the process of creating and modifying the policies which will be assigned to each of the protected VMs/appliances. We’ll start by building out the policies using the base ones installed by as part of the initial Trend installation as the parent/base policies. It is recommended to turn off all modules within the Base Policy and only turn on the features at the required sub-policy.

Each OS system should have it’s own policy based on the parent policy. These include RedHat Linux, Windows 2003/2008/2012 etc. and within those the various versions that require different exclusion lists such as General Windows Server, MS Exchange and MS SQL.

Modify the modules as required within the policy. Enable Anti-Malware to be On. When the DSVAs are deployed later and registered with vShield Manager this will appear as Real Time. Until the DSVAs have been deployed this will just appear as On. It will automatically change to Real Time once the DSVAs are deployed in a few later steps. 

Click on Anti-Malware and remove the Inherited scans and schedules and select from the scan configurations that were created above for the relevant system. Once you have selected the required scan configurations click Save and proceed to the next step.

Click Save. The policy will then be selected when a machine is being protected.

Step 11 – Install the Relay software on DSM

Now that the core components of the environment are configured and built out the next step is to get the relays up and running.  The software for the Relay needs to be copied across to the DSM server via WinSCP. I had done this earlier. Change to the folder where the software has been installed and run the following command:

rpm -i Relay-RedHat_EL6-9.0.0-3044.x86_64.rpm

Once the software has installed you will see the ds_agent being started

Once the relay software has been installed go back to the web console and select Administration -> System Settings -> Updates and select View Relay Groups. Select New Relay Group and select Download Updates from Source

You will get the option to add a member to the relay group when you activate the DSM in the next step

Step 12 – Activate Deep Security Manager

The next step is to activate the DSM. Within the web console go to Computers and the folder that you have allocated for TrendMicro devices. Select the DSM and from the actions list select Activate/Reactivate

Once the DSM has been activated a relay group needs to be assigned.  This is as easy as selecting a checkbox within the Relay group from above.

Each DSM needs to be assigned a security policy which should have been created earlier. I created one for Deep Security Manager. Double-click the DSM name under the Computers tab in the web console and from the drop-down select the required policy and save the change.

The Relay on the DSMs is also an agent so Anti-Malware protection is enabled to protect the DSMs.

Step 13 – Prepare ESXi Host

In order for Trend Deep Security Manager to be able to manage the VMs and appliances that sit upon the ESXi hosts they must first be protected. This involves installing the filter driver for Trend as part of the ESXi preparation. Be aware that this will require a reboot of the ESXi host and it will also put the host into maintenance mode during the preparation stage so it’s best to do that in advance of running Prepare ESXi from the web console so you can better manage the migrations off the host. In Computers go to the Hosts and Clusters folder and select the ESXI host. Right-click and select Prepare ESX.

Communication between the ESX server must be possible to the DSM. When you get this screen click Next

Next you will be reminded that the ESXi host will need to be rebooted as part of the preparation and filter driver installation. Select Yes and click Finish

This will then begin the preparation of the ESXi host which installed the filter driver on the ESX host and also configured the vSwitch to use the vmservice-vswitch and creates a Virtual Machine port group on the isolated network created earlier as part of protection in vShield Manager

Within vCenter you will see in the tasks screen the port groups being created and the command for the host to be rebooted.

 

The last part of the preparation will involve being asked to deploy a Deep Security Virtual Appliance. In my case I decided not to do this straight away and to do it later.

Once the ESXi host has been rebooted go to the Configuration tab for the host and select networking. You will now see a virtual machine port group called vmservice-trend-pg

Step 14 – Deploy and Activate Deep Security Virtual Appliance

Now that the ESXi host has been fully prepared you can now deploy the appliance if you selected No during the ESX preparation . Right-click on the prepared ESXi host and select Actions -> Deploy Appliance

On the next screen click Next. You will then be asked for an appliance name, the database to deploy the appliance onto, the folder that you want to place the appliance into and also what the Management Network should be. Once they have been put in place click Next to continue.

When configuring the details of the DSVA, you need to ensure that it is on the same management network as the DSM. Next you will need to enter the IP address, NetMask and default gateway. Re-enter the appliance hostname as per the appliance name from the previous screen and click Next

On the next screen you will be asked if you want to have Thin or Thick provisioned format. The preference should be toward thick provisioned. This will continue to build out the appliance and deploy it on the prepared ESXi host. When the DSVA has been deployed it will appear under the vmservice-trend-pg in the vmservice-vswitch

As part of the deployment for the DSVA you will be prompted to Activate the DSVA now. If you choose no then you can activate the DSVA by right-clicking the DSVA in the Computer listing and select Actions -> Activate Appliance. Otherwise click Next on this screen to activate.

When you activate the appliance you will be requested to select a policy to assign to the DSVA. I have created a specific policy for Deep Security Virtual Appliance and selected this from the drop down box.

The DSVA will then be deployed and activated

Step 15 – Configure DSVA

The first step in configuring the DSVA is to assign a Relay Group so that it can receive its updates. Right-click on the DSVA within the web console and select Actions -> Assign Relay Group. It is important to assign a relay group as this will allow the DSVA to communicate with the DSR and pulled down the anti-malware components needed to provide protection against threats.

Assign the Relay Group that is within the same network as the DSVA from the drop down menu. The DSVA has been successfully deployed and activated; by default the DSVA has the wrong time zone settings so these need to be modified. To login into the DSVA console the username and password is dsva. Please note, make sure that the DSVA has sufficient memory to protect the certain amount of VMs on a given host.

Select Configure Time Zone to modify the current time zone. Once the correct time zone has been selected it will take approx 10 seconds for the time to change on the appliance.

Note: Please note that affinity rules will need to placed for each DSVA for each corresponding host as the DSVA should not vmotion on to other hosts. This needs to be set under DRS on the cluster level and the details can quickly be found through a search online

Step 16 – Activating Windows and Linux Guests

To protect a Windows VM you can can quickly activate by doing the following:

1. Right click the Windows guest that needs protection and choose Activate
2. Right click the Windows guest again and choose Assign Policy

If the VM displays an error saying Anti-Malware engine offline you need to ensure that the vShield driver within VMware tools has been installed on the server.

For Linux guests, it is slightly different as they will be protected by a DSA. The process of activation is:

1. Upload the DSA file to the Linux guest via WinSCP
2. Install the DSA using the command: rpm -i Agent-RedHat_EL6-9.0.0-3044.x86_64.rpm

To activate the agent you can choose two methods:

1. Method a)
2. Right click the Linux guest in the DSM and choose Activate
3. Right click the Linux guest again and choose  Assign Policy and then Assign Relay Group

1. Method b)
2. Run the command to activate the DSVA to a DSM and apply the relevant policy: /opt/ds_agent/dsa_control -a dsm://DSMName:4120/ “policyid:3” “relaygroupid:1”

And there you have it, Trend Deep Security Manager will be up and running in your environment.