		Search on full text & attached documents :

Home page > ...

Element IV - Anti-Spam Technologies

Latest update : 20th April 2006

Element IV ELEMENT IV - ANTI-SPAM TECHNOLOGIES

Introduction

Spam presents complex technical challenges, and therefore solutions to eliminating it need to be supported by appropriate technical measures. While government action and legislation are fundamental, they are insufficient to meet the challenges posed by spam, viruses and spyware. In addition, the Internet structure makes it particularly difficult for enforcers to identify spammers, and therefore to punish them.

The nature of spam does not lend itself to easy definition. Despite problems of definition, however, there are both technologies and techniques that can be used to help control the problem of unwanted e-mails. This section is meant to provide a neutral overview of the various types of technological tools and methods as well as factors to consider prior to their implementation.[75] It refers specifically to tools as opposed to solutions. While technology is designed to address many of the problems created by spam, and may in fact “solve” some of the specific issues related to spam, an overall solution to spam can only be achieved through a multifactor approach that includes technology, policy (including regulation where appropriate), practice, and education.

Anti-spam tools operate at many levels – point of origination, in the backbone, at the gateway and on the recipient computer – and may be used alone or in combination. Updated information and resources are available on the Toolkit Web site at www.oecd-antispam.org.[76]

This section is addressed in particular to mail server managers, in order to provide them with an insight into the strong and weak points of each filtering technique, to enable them to choose software according to their e-mail policy and needs, depending upon their planned architecture. The focus of the section is on practices for incoming mail, while practices aimed at reducing outgoing spam have not been considered.

Tools that deal with spam need to focus on both the mail and on the behaviour surrounding the mail. In light of these multiple factors, many instruments and methods are based on sets of rules or assumptions that work alone or in combination to identify suspect e-mails. Over time, spam has grown in scope to include more viruses and malware. This requires defensive technology to go beyond the text-based tools to tools that analyse behavioural and contextual factors in determining whether to accept or reject specific mail or even attempted connections. Considering the increased security threat presented by spam, we expect that anti-spam technologies will either contain more, or need to work in conjunction with, advanced security and authentication technologies.

The importance of tool/technology context

Some of the tools/technologies considered in this section are specifically designed to be implemented at the entrance to the e-mail platform, whereas others can be more usefully deployed after the receipt of messages but prior to delivery to the end user. At each stage of filter application, the aim of implementing a rule may be to refuse or reject the electronic message, or simply to mark it or deliver it to the end user’s spam box.
The relevance and usefulness of each rule can therefore only be judged in terms of the precise context in which it is applied, the level at which it is applied in the message distribution process, and what finally happens to the communication.

Combining tests

Judicious application of technology should be the backbone of any approach that aims to defeat spam. One should be aware that none of the technologies discussed in the following will act as a “silver bullet” or one-stop solution to the problems created by spam. Rather, all of the technologies are complementary and will be most effective when implemented in conjunction with each other. The integration of a number of technologies is necessary to reduce the harmful impact of spam on a system.

The attention of administrators is drawn to the fact that tests are not necessarily to be used in “all or nothing” mode. On the contrary, it is preferable to combine tests to maximise the number of spam e‑mails intercepted while minimising the number of legitimate e-mails inadvertently intercepted or refused.

All or nothing refusal – this is one possible response from services using a black list. Any message that fails the test is refused. The occurrence of the error does depend, however, on where the rule is located in the distribution process.
Access privilege – this is one possible response from servers using a white list. Any message that passes the test is accepted. There is no risk of a legitimate message being rejected, but there may be false negatives. For example, a domain white list is of no real interest if the sender’s domain is not authenticated (with Sender Policy Framework or DomainKeys Identified Mail, DKIM).
Many spam messages or worms claim to originate from recognised consumer brands in the hope of gaining access privilege.
Scoring – this is how programs such as SpamAssassin (and also procmail) combine their tests. Avoiding the inconveniences of “all or nothing”, scoring is highly recommended. However, it is costly in terms of machine resources and the continued requirement of updating scoring factors to maximise hits while minimising false positives.

The conventional method is to run several “all or nothing” tests and then score the messages that have been allowed in.
The recommended good practice is to combine several techniques, but not so many that the resultant complexity creates an unreliable mail delivery system.

Types of anti-spam technologies

Authentication of electronic mail

The first comment that needs to be made is that mail authentication methods fall into the category of rules, which, although they help in the fight against spam, do not constitute specific anti-spam technologies.

An analogy may help to make this clear. Identity cards are not a “trust marker” in that perpetrators may also have an identity card. The requirement for transparency, however, will be of greater benefit to legitimate senders than to spammers.

SPF and/or Sender-ID

A major force behind the proliferation of spam is the ability of spammers to hide the true return address of their messages. The architecture of electronic mail does not imply a prior contact between the sender and the recipient. Therefore, it is not possible to rely on systematic authentication. The problem is of growing concern because forged addresses have been used in phishing scams that lure message recipients into disclosing credit card numbers and other personal information.

The application of this technology is still emerging and therefore lacks standardisation, but authentication works by flagging or blocking e-mail messages whose true senders cannot be verified. The key advantage of sender authentication is that it places the burden of spam dissemination on the sender rather than on the receiver and renders phishing attacks more difficult. The increased costs for senders are offset by guaranteed message delivery if senders are authenticated and are using the system legitimately. The specifics of the verification process vary with the model chosen, and several server authentication models currently exist. Two of the most prevalent are Sender Policy Framework (SPF) and Sender-ID.

These two techniques can be discussed together because they share several common features. The question of which one to choose, however, is less straightforward.

SPF and Sender-ID can be used to test whether an e-mail server is authorised to send on behalf of a given domain. This is done by publishing a record in the Domain Name System (DNS), which lists the authorized e-mail servers for a domain. The two techniques primarily differ in the choice of the identity tested. SPF tests the envelope’s MAIL FROM (2821), while Sender-ID tests the headers (2822).

Server administrators take two types of action - they publish SPF records in the DNS and they test them on entry.
Few domains currently publish SPF records, with the result that interest in it is limited. Some administrators note that Sender-ID is a new concept and has not yet been widely tested. The two techniques are not yet based on a stable standard.

The authentication of electronic mail by checking the IP addresses of the sender’s server will help to reduce and manage spam in the future. This will probably call for the creation of services above authentication, for example private white lists, reputation services, and accreditation services.

Issues for consideration:

SPF and Sender-ID can be tested on entry. Because these are emerging tools, users are cautioned against too much reliance on them until further uptake, unless otherwise indicated by the remote server or the particular context. Authentication is a critical component in any mail security procedure. SPF is currently the most widespread technique. Publishing SPF and Sender ID records is also recommended.

DKIM /or META

DKIM and Message Enhancements for Transmission Authorization (META) are used to authenticate the sender domain by means of a cryptographic signature automatically added by the e-mail server. At present, the immediate practical benefits are very low since few domains sign their messages. Furthermore, administrators will note that these three techniques are not yet based on a stable standard.

The authentication of electronic mail by cryptographic signature of the message should help to reduce and manage spam in the future.

DKIM is the most publicised of these models. The model works by requiring a digital signature, or private key, on all outbound messages. The incoming messages are authenticated at the domain and mail server levels by ensuring that the private key matches the public key already on file. This method ensures that the message could only have come from the originating ISP.

Issues for consideration:

These techniques can be studied and tested. It should be noted that they are emerging technologies, and, pending further large-scale deployment, it may be premature to rely on them at the present juncture.

Existence of the sender’s domain and eliciting a response

Many spammers send mail with a non-existent sender’s address. A rule can be used to refuse these messages, such as the Postfix directive reject_unknown_sender_domain or the j-chkmail directive BadMX. Another possibility is to verify the validity of the record for the incoming server (MX) for the domain given in the “from” field of the message. Some spammers set up a dummy MX record to avoid angry replies of protest (for instance, the MX goes to 127.0.0.1, which means the local sender).

These rules call for a small amount of DNS traffic, which probably would have occurred anyway during the reply, and they can also reject a certain amount of spam.

Issues for consideration:

This technique is not widely used. The annoyance is minimal in that, if the sender’s domain does not exist, it will be difficult (although not impossible, thanks to the automatic function of Reply-To) to reply to the message. “No reply” send boxes should not be used.

Existence of a Pointer Record (PTR)

A PTR record of the DNS can be used to translate the IP address of the sender’s server into a name, although without necessarily checking that this name is consistent with the sender’s domain.

The addition of such records is not always under the control of the sender’s domain (if there is no addr.arpa delegation by the IP, for example), which, even if it is legitimate, may be unable to meet the obligation. These records can be used to determine the source of an e-mail message and whether or to what extent it can be trusted. They can also be used to determine whether a mail originates from a residential IP address or to redeliver an error message to the right server.

Issues for consideration:

This test can prove useful in order to indicate in the header of the e-mail that the field is empty or suspect (for example it did not lead to an IP when directly requested). The PTR field can still be useful (particularly if the remote server agrees) to return certain information to the recipient. However, using this test to refuse an electronic message for the sole reason that the PTR field of the connecting IP has no content should be avoided. The utilisation of this test to refuse an electronic message should also be avoided unless circumstances (e.g. if the field can be used to determine the origin of the message) or policy (e.g. refusal of messages from particular residential IPs) dictate it.

Blacklists/Whitelists

Traditional filtering as well as tracking complaints across user communities can ultimately lead to whitelists of acceptable senders and blacklists of suspected spammers. The whitelist/blacklist approach is often a too drastic solution to be acceptable by most users. Whitelists are time-consuming to create and will require continual updating. Blacklists require similar monitoring. All lists need mechanisms and procedures for updating to address false positives and fraudulent complaints to a listing. Spoofing and open relays can also create issues related to the appearance that mail has originated from a source.

Blacklists are based on the principle of listing sources of spam. This list can include the names of machines, IP addresses or electronic addresses. It can be implemented by an entity for shared use, or introduced and maintained by the server using it for its own requirements.

With current Mail Transfer Agents (MTAs), this test can be carried out in the SMTP session and therefore result in rejection even before the message is sent. Some lists contain open relays that do not send spam alone. Their open relay configuration can be treated as illegitimate behaviour by the platforms to which messages are sent.

The quality of blacklists varies enormously, depending on the professionalism of the compiler. Many lists are poorly managed, abandoned, or of dubious integrity: names can be added quickly, the applied criteria may be unclear, and the removal from the list may be virtually impossible or be operated only on a payment basis. This problem is mainly due to the absence of a code of conduct or any kind of regulation to discipline and limit the functioning of blacklists. If this solution is to be used in the future, a co-operative effort to establish a list of good practices, clearly establishing cases in which addresses can be blacklisted and the conditions under which they will be removed from the lists, is necessary.

Blacklists will inevitably contain inaccuracies that will prevent some legitimate messages from getting through to the consumer. This problem, known as the false positive problem, has led to legal disputes when legitimate senders believed they were erroneously placed on an ISP’s black list. Further, the false positive problem for individual users can result in a serious drawback of relying solely on traditional filtering technology to stop spam.

Although their utilization raises many concerns, blacklists are a quick solution to refuse a connection to machines whose behaviour endangers the security or quality of services of the platform to which mail is sent, or to reject messages from certain senders.

Issues for consideration:

Care should be taken in choosing lists. Before choosing a list, it would be advisable to consider a few points:

Use lists that are well managed.
Ensure that there is a professional administrator who replies and reacts.
Addition and removal criteria are clear, reasonable and accessible.

When an external list is used, its activity needs to be monitored because quality evolves over time. If an administrator manages his own black list (which is resource-intense), the above criteria for quality will obviously also apply. Since it is difficult to provide a rapid summary of black lists, given their variety, a cautious approach has to be recommended.

Address of the sending server treated as either “dynamic” or “residential”

This is a particular form of blacklist in which the criterion for addition to the list is the fact that the IP address being blocked corresponds to the machine of an individual subscriber to an ISP and not to the mail server of an organisation. The idea is that an ordinary subscriber does not send mail directly in SMTP, but passes through the PTA of his provider. This typically means the machine being blocked is directly sending spam messages from a spammer, or more commonly that the messages are being sent without the owner’s knowledge (i.e. the machine has been compromised and turned into a “zombie” in order to send the messages).

The lists of such addresses are not always reliable since most of them have been compiled using heuristics, such as the presence of “adsl” in the name of the machine. Managing such lists is also resource-intensive.
In contrast, some of these lists, notably those compiled by the server using them, can be used to distinguish between servers authorised for a domain and the residential lists. Moreover, some domains publish the ranges of residential addresses for their domain.

This test can be seen as discriminating between “pure consumers” and “providers”. The latter consider legitimate the policy by which the owner of a domain refuses to connect his machines to residential addresses, as these are currently the main source of spam. Consumers however argue that spam exists and the freedom to use e-mail must be protected.

Filtering

Filtering is the most common technical anti-spam technology. The main benefits of filters are the ease of implementation and the flexibility that users have in deciding which messages should be treated as spam. Heuristic filters require that users specify criteria, such as keywords or a sender’s address that will prompt the filter to block certain messages from reaching the consumer’s inbox. Spammers who deliberately misspell words or spell them in a different language easily outsmart the keyword approach. Bayesian filters are based on experience. They create statistics about the messages in a recognition table for future reference for individual users to distinguish between spam and legitimate mails. The filter then lets through only messages that resemble the user’s previous legitimate mail.

Heuristic filters

These filters are based on the principle of testing for the presence in the message of certain typical features of spam, such as the exclusive use of HTML or the type of customer to whom the mail is sent. The test is weighted through a learning process based on a set of known spam mails and a set of mails known to be legitimate (the scores are therefore not calculated by a human in order to reduce subjectivity). One of the most well-known and widely used filters is SpamAssassin. Yahoo!, Hotmail, and AOL, among others, use heuristic filters in their junk-mail filtering systems.

These filters carry the risk that a message using spammer techniques – spectacular messages in HTML, for example – will be classified as spam. Furthermore, it should be noted that the filters use large amounts of machine resources.
These filters can detect a high proportion of spam mail, and they do not need to be taught or configured. However, since they use a large number of tests, it is best to be aware that it is possible to change which tests are run and the scores used to classify messages as spam.

Issues for consideration:

Easy for users to install, these filters offer a high degree of filtering for little work by the administrator. Care must be taken to regularly update the test base, as in the case of anti-virus programs. A two-year old anti-spam program may hardly catch any spam as the latter has now evolved. Potential for false positives must be monitored, and filter criteria must be adjusted. Also, some of these filters implemented at the B2B level have the ability to provide notice to the recipient of certain suppressions to allow a review of whether it was properly classified.

Keyword filters

These are binary filters that search for a keyword (“Viagra,” etc.). The risk of false positives is very high and the ability to avoid these by spacing, alternate characters and misspelling is also substantial.

Issues for consideration:

Keyword filters lack granularity and effectiveness unless they are specifically relevant to the context in which they are used.

Summary or fingerprint filters
Fingerprint filters, such as Razor, construct a fingerprint of the message submitted to them and indicate whether it has already been identified as spam. There are many false negatives because a number of types of spam mail are not identified even when the server scans them with Razor. Furthermore, the message sometimes varies sufficiently for it to generate a different fingerprint. One solution to this problem is to delay the mail (as greylisting does). They generate few false positives.

Bayesian filters

By way of an opening comment, it should be noted that the techniques described do not concern personal configurations on end-user machines but rather those of a group server.

The principle on which the Bayesian filter works is to prime its engine by examining a set of known spam e-mails and a set of e-mails known to be legitimate, then after teaching itself the vocabulary used by spammers from this known list it will use Bayesian probabilities to calculate whether a message is spam. In the case of a group filter, the learning is usually conducted by the system administrator.

Being based on the concept of spam vocabulary, these filters can pose problems when used on a shared basis. In a small-scale and highly uniform environment (for example a firm or a University department in which everyone works in the same domain with similar vocabularies), this may be acceptable. However, this would undoubtedly not be the case for a major e-mail provider and particularly a public provider unless the group base offers each individual user the possibility of customising the filter for his/her own mailbox. The problem is that what is acceptable vocabulary to some users may trigger the filter if it has been deemed by the group to be spammer vocabulary.

Despite potential issues at the group level, these filters are highly effective when used by individual users and are one of the few solutions which, when used alone, can filter out almost all spam mails after suitable training. Microsoft provides customisable Bayesian filtering in Outlook and Exchange Server. Another example is the free program Bogofilter (www.bogofilter.org).

Issues for consideration:

These filters should be used on an individual basis or allow users to customise them.

Behavioural filters

This type of filter examines the behaviour of the remote server, such as the number of mails sent by unit of time. Rate limiting is one example of this type of filtering. The idea is that ordinary mails are only sent individually or in very small numbers and spam mails are sent in very large batches.

This type of filter is extremely delicate because typically there is no way to distinguish between a spammer and a legitimate distribution list server, such as a newsgroup.

According to some experts, it is nonetheless legitimate for a platform to refuse certain volumes of mail, primarily due to its size or its mission to ensure the security of its networks. It would also seem legitimate to ask bulk mail senders to respect the resources of the remote platform by bearing the cost of distributing their messages without trying to send them too quickly in order to free themselves from the costs inherent in using e-mails as a channel of communication.

Issues for consideration:
Depending upon the policy of the recipient, this type of filter can be used either to restrict flow rates or to block incoming mails. In the latter case, an explanation of the reason for the reject must be accessible by the sender so that the latter can comply with the distribution criteria.

HELO/CSV

A sending computer identifies itself by name to a receiving computer at the beginning of each SMTP transaction. The SMTP command the sending computer uses to identify itself by this name to the receiving computer is called the "EHLO" or "HELO" command.

Certified Server Validation (CSV) is a service that provides a mechanism for a mail-receiving server to assess a mail-sending server. It builds upon the existing practice of service providers that accredit the networks from which sending systems are connecting.

HELO tests check that the remote MTA is properly configured, but these tests do not indicate whether it is a spammer or not. CSV tests add a probability test on the name: does it really correspond to a domain? Unlike SPF or DKIM, CSV does not authenticate the domain sending the message but rather the domain of the e-mail server (which may be different, for example, in the case of a provider serving a large number of customers).

Configuration directives — for example the Postfix directive reject_invalid_hostname — test the name announced by the server. Using conventional HELO tests results in a very high number of legitimate messages being rejected. However at the moment only few sites know how to modify HELO to make it work properly. This is probably going to change in the future since a growing number of sites will test HELO, thus creating an incentive to improve it.
Issues for consideration: These solutions may be considered when they concern the configuration.

Greylisting

This is the deliberate sending of an SMTP 4xx error code (a temporary error as opposed to a 5xx definitive error, see RFC 2821) when encountering a new sender. The latter, if it is a normal MTA, will try again later (usually 15 minutes later) and its message will then be accepted. Most spam software programs do not make multiple send attempts. This technique is highly effective and blocks all spam mails that are not sent through an open relay or by the MTA of a provider. It prevents receipt of certain messages from poorly configured servers and lends itself particularly well to be used in conjunction with a whitelist.

Issues for consideration:

This technique is simple to implement but its future remains uncertain: if everybody adopts it, then spammers will start sending more messages again. Use of this could give techniques such as fingerprint filters time to show their merits before this method becomes less useful due to spammer adjustments.

Tokens/passwords

The aim of these techniques is to include a password in the address to which the e-mail is sent or to use a challenge/response system such as the Turing test. The spammer’s software will not know this password and will be unable to pass the test.

These techniques have no false negatives — unless spammers decide to employ thousands of people at very low labour costs to do the work.

A certain number of legitimate users will refuse to or will be unable to pass the test. There will therefore be many false positives. These techniques are only of interest to highly popular recipients who already receive vast amounts of bulk mail, including legitimate mail, or to any recipient who wants to reduce the number of messages received, which falls within the scope of freedom of communication. It is necessary to be aware that not every sender will accept the test imposed. Educating users about the merits of this technology and taking the test may help reduce the non-acceptance rate.

Issues for consideration:

These filters are not so much a filtering technique as part of a whitelisting policy. Users of such filters must be aware of this fact. Please refer to the above paragraph.

Various techniques

This section covers various techniques mostly experimental or insufficiently tested.

Envelope tests (BATV, SES)

These techniques are recent developments and insufficiently deployed to be taken into consideration.

Certification of Bulk Mails - Sender Reputation.

Although effective sender authentication will give Internet Service Providers (ISPs) a much more straightforward task when dealing with spam, authentication is only a preliminary step toward eliminating spam. Once the sender can be identified, factors such as reputation and accreditation are needed to determine whether a message should be classified as spam before it reaches the user. Independent authorities would manage the certification process and set the criteria. An oversight board, with cross-sector representation, would oversee the certification authorities.

Toward this end, the Trusted Email Open Standard (TEOS) has been created by the ePrivacy Group. TEOS grew out of ePrivacy’s industry self-regulation program that aims to separate legitimate e-mail from spam. TEOS goes beyond authentication and creates a trusted identity for e-mail senders based on signatures in e-mail headers. Unlike the authentication signatures of DKIM, the TEOS signatures are visible seals in messages, certifying that the sender meets specified criteria.

There are a number of other organizations that are developing sender reputation schemes. Both America Online (AOL) and EarthLink use SPF to verify their outgoing e-mail. Brightmail and Microsoft will be using Brightmail’s model to conduct joint testing on the efficacy of sender reputation systems. Further, Sendmail, a provider of secure e-mail systems, will be collaborating with Yahoo! to test DKIM.

Issues for consideration:

Unlike the authentication service, which is more empirically based, reputation and other services may have a larger subjective component provided by the rules/practices of the supplier. How their services will work and the credibility/utility of any particular service offering will have to be proven over time.

In order to reduce the problem of bulk e-mails erroneously filtered as spam, industry continues to discuss the efficacy of a bulk mail certification mechanism. For example, legitimate bulk mails could be identified at the ISP level with a label that is recognised by the server, thus enabling more confident use of e-mail filters. Several criteria could be used as input to the certification process, such as a commitment to strong privacy practices. For instance, France is working with its data protection agency (CNIL) towards a certification for senders who notify the use of client records.

Each ISP would maintain a whitelist of the certified clients. The proposal requires an agreement among ISPs on the certification process and involves no external intervention. However, the method would require a critical mass of ISP participation to be effective and would be based on trust among ISPs, since there is no external oversight of the certification process. In addition, assigning a fixed number to the definition of bulk mail may be problematic. Crafty spammers could use multiple free e-mail accounts to send large quantities of spam, with each account sending a number just under the pre-defined bulk mail threshold.

Micro payment systems

Conceptually, the simplest solution to the spam problem is requiring payment for each e-mail message that is sent, just as with conventional mail. Such a solution could be accomplished through structural changes to SMTP and would create a system in which a message was guaranteed to be delivered instead of suffering an ISP rejection on non-standard suspicion of spam. Money is the most obvious currency that could be used for payment but other ideas have been proposed.

For example, Microsoft has advocated computational spam-fighting, whereby each sent e-mail is paid for not in money but in computational time. Microsoft has proposed increasing the computational time to send each message to about ten seconds for unsolicited e-mail. This plan would have little effect on the casual e-mail sender, but those sending millions of messages a day would have to invest heavily into additional computational resources, a significant deterrent to sending spam. With this approach, users may choose to keep a whitelist, meaning that the list members would be exempt from the computational fee.

Another example is IronPort, whose Bonded Sender program is a whitelist certification program that requires a bulk e-mailer to post a monetary bond to gain accreditation. If the sender violates the acceptable e-mail practices, a debt is taken from the bond. Bonds range from USD 500 to more than USD 4 000, depending on monthly mail volume. Infractions, and bond debits, are based on user complaints. Microsoft has partnered with IronPort and will use the Bonded Sender program for incoming mail to its MSN and Hotmail accounts.

Does the sender’s server reply if you try to respond?

There are no particular recommendations about using this technique.

PGP signatures

This technique is far too uncommon at present to be taken into consideration.

System configuration

Industrial and individual-level security best practices for ports, firewalls, networks, routers, proxies, access, passwords, permissions key protection and software installation are examples of use of system configuration as anti-spam technology. By configuring your system to block unwanted mail, one only captures some percentage of spam is captured. However, as more and more systems install these mechanisms, spammers will certainly become more ingenious, but it will also become less and less desirable to spam as there will be more obstacles to overcome. People spam now because it is simple, quick and cheap. As that changes – and hundreds of thousands of system administrators are working to change that situation – it will be harder to spam successfully.

Anti-virus tools

Anti-virus tools are important technologies that reduce the risk of spam e-mails infecting computer systems. Generally, harmful spam e-mails have potentially virus-initiating files attached. Anti-virus software can scan mailboxes and prevent virus infections.

Some ISPs are working to constantly monitor and update anti-virus API (application programming interface), VSAPI, with Exchange Server. This technology provides anti-virus scanning on user mailboxes to put scanning out to the network edge reducing the impact of viruses and virus-tainted e-mail on network infrastructures. It is also possible to prevent infected e-mail from leaving an organisation by scanning outgoing mail, in addition to incoming mail. The Clam Antivirus system http://www.clamav.net/ is an example of free software doing that same job.

Anti-spyware tools

Spyware has emerged as a major threat to enterprises because of the covert way it is installed to gather user information and deliver pop-up advertising. Spyware programs and keystroke loggers also have been used by malicious hackers to hijack e-mail addresses, user passwords and credit card information. Overall, spyware slows down computers, hijacks Web browsers, and sends personal information to advertisers. Anti-spyware tools can also detect potentially harmful spam e-mails, which in turn can assist the reduction of spam e-mail.

How to use this review of technologies and factors to consider

The utility of any tool(s) will be dependent on the needs, technical ability and the infrastructure of the user of the same tool. Tools are meant to be deployed at different parts across the system and for differing purposes. Users will have to consider their needs and strategies of defence in depth as they choose and deploy anti-spam tools. Tools themselves vary in maturity, efficacy, reliability and deployment. Some tools are more prone to false positives some are more effective in certain areas and some have greater overhead in terms of cost, infrastructure, bandwidth/capacity and needed technical expertise. A number of these factors have been listed for consideration, but user will have to gauge tools in the specific context of their contemplated application.

Some of the above tests are designed to fight spam, while others aim to prevent certain types of behaviour which pose a threat to security and fail to respect the resources of the platform to which mail is sent or simply do not comply with the accepted rules for sending electronic messages. When the rule is implemented after the receipt of the data constituting the message to be delivered, it remains to be decided how the message should be dealt with. This will obviously depend upon the results of the tests carried out. Some tests are more reliable than others and can therefore justify recourse to more drastic measures. Furthermore, it may be decided to carry out other and more expensive tests on certain messages.

The various options for dealing with a message depending on the location of the rule implemented are presented below.

Rejection in the SMTP session

The interest in such rejection lies in not taking charge of the electronic message, whose distribution remains the responsibility of the remote server, which has been advised of the situation. In addition it saves bandwidth capacity, firstly because the message is not received and secondly because the remote server will not have to send DSNs (Delivery Status Notification, the message generated in response to a rejection, see RFC 3461) that the message might generate. The task of issuing such a non-delivery message is transferred to the sender.

However, this type of rejection means that it is not possible to keep a copy of the message (and therefore to retrieve a legitimate message that might not have been accepted or simply to investigate a rejection).

Moreover, not all SMTP servers are currently able to run certain tests during the SMTP session. This is changing, however, with the increasingly common use of new products and in particular interfaces such as sendmail’s “milter”, the Postfix “policy server” or the future OPES which will be able to connect any program to the SMPT session.

Issues for consideration: Such rejection is advised.

Silent rejection

This method often confounds regular users who expect their e-mail to be delivered or at least to be told that it has been rejected. The “deliver or advise” alternative is a cardinal principle of e-mailing, but one which will probably have to be abandoned due to the number of “joejobs”.[77]

Ideally, a record should be kept of e-mails destroyed in this way so that techniques such as Message Tracking can be used, for example by deploying RFC 3885 describing the message tracking protocol, which allows users to learn what happened with their messages (like the parcel tracking systems of typical parcel delivery companies).

Issues for consideration:

It should be noted that all “message losses” of the type complained about by users are not necessarily silent rejections by a provider. The cause of the loss may lie elsewhere (premature deletion of a message by the human recipient or deletion of a DSN by the human sender).

Rejection by sending a DSN (Delivery Status Notification or “bouncing”)

This is the method traditionally used in Internet e-mailing. However, due to the presence of “joejobs”, there is a risk of penalising innocent senders, as may be seen with the anti-virus programs which mistakenly send DSNs.
Issues for consideration: If a message has been classified as spam, its sender’s address is probably false and there is therefore no point in sending a DSN. Conversely, some server managers do not want to run the risk of failing to inform the sender that the message has not been distributed. It is difficult to recommend what approach to adopt in view of the advantages and disadvantages of each of these solutions.

Delivery to a spam box

When few messages are blocked on entry to the platform, the spam box can contain very large volumes of messages, which can discourage users from reading it. The message is not destroyed, but the user is given an opportunity to remedy false positives.

Issues for consideration:

This is a very useful complement to blocking on entry to the platform. The user can implement this technique to supplement the measures taken by his e‑mailing provider if the latter does not offer a spam box in its anti-spam package.

Marking

The server takes no decision but simply places a note on the e-mail. This technique gives the user full control, but will also force the user to download spam mail.

Note that an e-mail service provider can offer the user the choice of simply marking the e-mail or delivering it to the spam box. It is relatively simple to manage.

Issues for consideration:

Marking the e-mail which modifies the message text or the Subject field is hazardous from both a technical standpoint (because it breaks signatures) and a legal one. It is preferable to add a header (such as “X-Provider-spamscanner: YES”) that all e-mail software programs know how to use to filter mail.

This section contains discussions of the various anti-spam technologies and their capabilities presently available, as well as of the methods to be employed when spam is received. Any attempt to combat spam effectively must involve the sensible administration of a number of these technologies in concert. None of the above methods will be entirely successful in isolation. When a number of anti-spam technologies are effectively used in collaboration with one another, the effect can to drastically reduce the level of spam impacting a system.

___________________________________

[75] This section is the result of a preliminary study carried out by the French anti-spam contact group. It was developed by different actors, including Internet experts and civil society representatives. The study was then enhanced thanks to the contributions of BIAC and of the US before being contributed to the OECD Task Force.

[76] This section presents a variety of technologies by type or category and at times may highlight a specific application in a category; this is not by way of endorsement but merely to illustrate the example. In addition, considering the rapid development of technologies, this section should be taken for an overview of technical measures available at the time of its preparation (second quarter 2005.

[77]An e-mail which pretends to come from someone that has never actually been involved.


About \| Contact us \| Terms & conditions \| Privacy policy © OECD. All rights reserved. Web site developed by the MDD with Spip 1.7.2 and Exalead.