HMAS Sydney
Introduction
This submission was provided to the recent Parliamentary Inquiry into the circumstances of the 1941 sinking of HMAS Sydney by the German raider Kormoran. Visitors wanting to know more about the early history of Australian Sigint - as well as those curious about the fate of HMAS Sydney - may find it of interest.
Defence Signals Directorate
Defence Sub-Committee
Joint Standing Committee on Foreign Affairs, Defence and Trade
March 1998
HMAS Sydney
The purpose of this Submission is to provide the Inquiry with a description of the conduct of signals intelligence (Sigint) at the time of the sinking of HMAS Sydney, and an explanation of the involvement of DSD in the management of Sigint-related records dating from that period. It seeks to assist the Sub-Committee in reaching a determination with regard to claims that certain records relevant to the loss of HMAS Sydney have been withheld or destroyed. This Submission, therefore, addresses only the first two of the Inquiry's Terms of Reference.
In essence, the Submission explains that –
- The Directorate has conducted thorough searches of its own records
and those submitted to it for inspection, and can confirm that it holds
no material relating to the Sydney which is not available on
the public record, other than three administrative files raised in the
1990's which deal with requests for information from members of the
public, and Questions on Notice.
- However, DSD has identified a number of references to the vessel in
material that was examined prior to its public release. These latter
records are all open for inspection at the Australian War Memorial and
Australian Archives repositories
- Certain records dating from the early years of the War, now in the custody of the Australian Archives, are subject to DSD-imposed exemptions from the 30-year rule prescribed by the Archives Act. These documents contain no references to the loss of the Sydney. Because the security issues that led to the imposition of the exemptions have changed over time, DSD is prepared to lift the exemptions, subject to confirmation from the US and UK authorities, to enable public inspection of the records in question.
The Sydney and wartime Sigint
The bulk of this Submission addresses the question of the post-WWII management of Sigint-related records. Before turning to that issue, however, DSD believes it may be of assistance to the Inquiry to briefly outline the nature of Sigint activity as it would have been carried out at the time of the sinking of the Sydney. This is because, as the Sub-Committee will be aware, a number of statements and claims have been made over the past few years that speculate on the role that Sigint played, or might have played, in connection with the Sydney-Kormoran engagement. DSD does not seek to canvass these particular matters in detail, since that is most appropriately left to the Inquiry. But a general understanding of Sigint at the time of the incident may assist the Sub-Committee in assessing the viability of some of the claims before it, particularly in relation to the kind of activity in which the WWII units would have been engaged, and the nature of the information that may have been contained in WWII Sigint-related records.
When issues of contention arise as to the WWII role of the Allied Sigint organisations, there is sometimes a tendency, on the part of some commentators, to assume that Sigint must necessarily know the whole story, or have access to all the relevant sources. Given the security constraints that must, of necessity, be observed in regard to the release of Sigint information, even after the events in question have passed into history, this is understandable. Unfortunately, it is also misleading, particularly as far as the case of the Sydney is concerned.
During WWII, and more especially during the first two years of conflict, the practice of Sigint would have been a difficult, demanding and very uncertain business. The main communications targeted by the Sigint units would have been sent by radio, most commonly in the High Frequency (HF) portion of the electromagnetic spectrum, which is best suited to long-distance radio communications. The airwaves would have been very heavy with message traffic, sent in many languages, and with a multitude of enemy and Allied military and civilian transmissions crowded into narrow segments of the spectrum. The task of identifying particular frequencies used by specific enemy targets among the many thousands of signals audible to the Sigint operators required a strong and dedicated effort by highly-skilled specialists.
Once enemy transmissions had been identified, a painstaking analytical effort was required to identify the ship or army unit originating the traffic. Both the Germans and the Japanese were experts at disguising the identities and locations of their combatants – for example, by encrypting the radio callsigns that identified the units concerned – and a range of special techniques had to be deployed in order to defeat these communications security measures. Adding to the difficulty was the fact that HF communications are subject to many kinds of interference, both natural and man-made, and can be seriously affected by adverse atmospheric conditions. During WWII, it could never have been expected that every message sent by a particular naval platform could be intercepted, even if the frequency used by the vessel were known to the Allied Sigint operators.
One of the greatest challenges for the WWII Sigint units was the widespread use of encryption systems and devices by the Axis forces. All combatants were aware that their enemies were listening to their communications, and went to great lengths to conceal their messages behind the strongest cryptography available. Not all German and Japanese cyphers were broken, despite the best efforts of Allied cryptanalysts. Many were, and the successful exploitation of those communications contributed significantly to the ultimate victory of the Allies. But the effort required was very great, and for Sigint to have a reasonable expectation of success against such difficult targets, a large organisation and substantial numbers of personnel was needed. Australia did not possess such a Sigint organisation at the time of the loss of the Sydney (other than a small RAN unit egaged in an effort against the Japanese).
It is important to recognise this context of WWII Sigint activity in order to realistically evaluate some of the theories that have been advanced in connection with the loss of the Sydney. For example, there have been claims that the Kormoran transmitted signals to Tokyo or Berlin after the engagement with the Sydney. DSD has sighted no record that suggests this was the case. But the Inquiry may wish to note that even if such transmissions did occur, there could have been no guarantee that a Sigint listening station would have picked them up. Even if they had been, the Allied units would not have been able to decypher them. The German Naval units used the Enigma device to encypher their message traffic, and most used what was known as the 'home' key, which was broken early in the War. However, armed merchant cruisers such as the Kormoran used the 'foreign' key, which was never broken. It should also be noted that Sigint would not have collected any signals transmitted by the Sydney itself; the Sigint effort was directed only at enemy communications.
The reporting procedures employed by the Sigint units need also to be understood, in order to provide a context for the post-War handling restrictions that have generated some debate by those anxious to uncover all traces of information related to the Sydney. Sigint units were able to intercept enemy communications by tuning radio receivers to the frequency of the enemy transmitters, and recording the messages sent by the enemy operators. Messages were sent primarily in Morse code, either in plain language, or after having been encrypted. The Sigint units reported the content of the messages they intercepted, and would normally have kept copies of their reports for their own records. The information provided by the units would have ranged from vessel position reports to high-level diplomatic communications.
Sigint activity was carried out under conditions of very tight security to minimise the risk of the enemy becoming aware that its communications were being exploited, and subsequently changing its communication security practices to deny the Allies continued access to crucial message traffic. The reports generated by the Sigint units were distributed under strict secrecy caveats and special handling regimes, and were provided only to a very limited number of readers. These caveats remain in force until the national Sigint authority of the Allied nation concerned (in Australia's case, DSD) declassifies the reports in question, either in their entirety, or in part.
The WWII Sigint units
Against that general background, it may be of assistance to the Inquiry to know the status of the Australian Sigint effort as of late 1941, and the likelihood of any records relevant to the loss of the Sydney surviving from that time. Unfortunately – as far as the survival of relevant records is concerned – the situation is far from clear. There was not in Australia, at that time, a centralised Sigint organisation. This should not be surprising, since the major theatres of war at that time were in Africa and the Mediterranean. Sigint activity against the Germans was mainly undertaken by the British, although Australian Army and Navy Sigint elements provided support to the UK effort. One of the main listening posts was the UK's Far East Combined Bureau (FECB), located in Hong Kong, and subsequently withdrawn to Singapore. Australia had not been engaged in significant Sigint activity before the War, although the RAN had conducted some operations against the Japanese, in concert with the British.
It was only after Japan entered the War in December 1941, and Singapore and the Philippines were subsequently overrun, that Allied Sigint personnel and equipment were evacuated to Australia to join the small number of units that were conducting Sigint operations from bases in Melbourne and Darwin. This led, in mid-1942, to the establishment of two major Sigint units, Central Bureau (attached to the HQ of the Allied Commander of the South-West Pacific), and the joint RAN/USN Fleet Radio Unit, Melbourne (FRUMEL), which was subordinate to the Commander of the USN 7th Fleet.
But all this happened many months after the Sydney-Kormoran engagement. In November 1941, the Australian Sigint effort was fragmented. Army, Navy and Air Force each had some capability, but it was not co-ordinated across the three Services, and no organisation comparable to DSD existed. This makes the identification and tracing of records from that period extremely difficult. Although FRUMEL and Central Bureau kept full records their activities and message traffic, the earlier activities of the Australian Service units were not so well documented. DSD has not sighted any records from Australian Naval or other Service units from the period prior to the establishment of Central Bureau and FRUMEL that contain Sigint-related references to the Sydney.
The situation with regard to pre-1942 records of (non-Australian) Allied Sigint units is no clearer. It is conceivable that the Sigint units previously based in Asia may have brought with them to Australia some records of their activity prior to the abandonment of their former stations. In this context, the most important unit is the USN's Fleet Radio Unit (later re-constituted as FRUMEL), which had extensive experience in Sigint operations in Asia. (Central Bureau had no precursor organisation.) However, it should be recalled that the Fleet Radio Unit was evacuated at very short notice from its base at Corregidor in the Philippines to Java, and again from Java to Melbourne. It seems likely that at least some of the Unit's records of intercepted messages were lost or destroyed, since its personnel had to travel with minimum equipment and personal belongings, usually in the cramped confines of a submarine. In any event, DSD is not aware of the existence of any records of the Fleet Radio Unit prior to its relocation to Melbourne in early 1942.
Records examined by DSD
DSD was constituted as Australia's national Sigint authority in 1947 (or, more accurately, the Directorate's precursor organisation, the Defence Signals Bureau, later renamed the Defence Signals Division, was so constituted). As such, DSD became the releasing authority for Sigint-related records dating from the WWII period.
DSD does not have custodianship of any records other than those generated in the course of its own activities. However, it it is required to examine Sigint-related records held by other elements of the Defence Organisation and government Departments prior to their release. Any material that contains references to Sigint is passed to DSD, via the Defence's Chief Executive Officer, Archives and Historical Studies section, for examination. This is in accordance with standing security instructions for management of such records, which require, in particular, that WWII Sigint-related records be accorded the same special handling protection as current Sigint records, until such time as a decision is taken to declassify them.
The procedure adopted, in brief, is that DSD examines the material, makes a determination as to whether the records in question contain any information that has the potential, if publicly disclosed, to prejudice the conduct of current Sigint operations (whether by Australia or its allies), or would be otherwise unsuitable for public release. If no such information is identified in the records, they are forwarded to Australian Archives, or returned to the relevant Department, for retention. If some material is identified as unsuitable for public release, DSD has the power to seek exemptions for such material from the general rule that all government documents must be released after 30 years. Those exempted documents, however, are stored by Australian Archives, or by the Department concerned, not by DSD. Each request for exemption is accompanied by an unclassified Statement of Reasons setting out the basis on which the exemption is sought.
The grounds on which exemption of particular records from disclosure may be sought are prescribed by section 33 of the Archives Act. They include those records whose disclosure has potential to –
- damage to the security, defence or international
relations of the Commonwealth;
- constitute a breach of confidence; or
- unreasonably reveal information relating to the personal
affairs of a person (including a deceased person).
The Directorate emphasises that the criteria on which it bases its requests for exemption relate only to the potential for public disclosure of the records in question to reveal Sigint capabilities or techniques, or information provided in confidence by an Allied government or, in some cases, the names of Sigint personnel. In a very limited number of cases, such considerations apply even to records dating from WWII.
Over the past ten or so years, DSD has examined a wide range of material submitted for examination by the Services. The historian assisting the Inquiry, Professor Dennis, has been provided with full details of DSD's examination of such records. All material dating from the period of WWII has been cleared for public release under the 30-year rule, with the exception of certain records in the date range 1 May 1940 to 31 December 1941, which are detailed later (see 'Current exemptions'). As indicated, DSD only examines such records as are identified by other elements of Defence or government Departments as containing Sigint-related material. The most significant of the records so far examined by DSD, so far as their relevance to the Sydney is concerned, are shown in Table 1. All of these records are available for public inspection.
| RECORD | DESCRIPTION | LOCATION |
|---|---|---|
| Sigint-related portions of MP1074 series | Australian Commonwealth Naval Board (ACNB) 'In-Out' message packs;
various classifications Note: Exemptions imposed in respect of MP1074/8, 'Out' message packs, Box 1 (see Table 2 for details) References to Sydney: MP1074/-9, Box 12, #28/1434, dated 09/1945: Report that the German Naval attache in Tokyo believes information that the Sydney was towed to Japan was 'definitely incorrect'. |
AA Mebourne |
| MP1185/-8 | Navy Secret and Confidential files, Boxes 1-24 References to Sydney: File 1997/5/305: Mentions Sydney's Sigint role |
AA Mebourne |
| A5954 | Shedden – Box 213 – File 2: Court of Enquiry – HMAS Sydney Boxes 518, 558, 731 and 2400 contain references to the Sydney | AA Canberra |
| B5554 | FRUMEL: Technical Record – Ciphers and Codes (the 'Red Book')
References to Sydney: Gives account of FRUMEL's decryption of Captain Detmers' diary |
AA Melbourne |
| B5823 | Translation of Detmers' diary | AA Melbourne |
| B6121-165P | Sydney-Kormoran: Operations signals, November 1941 | AA Melbourne |
| PR88/178 | Kormorandocuments | AWM Canberra |
FRUMEL and Central Bureau records
DSD has also examined records from Central Bureau and FRUMEL. Full declassified records of these organisations are available for inspection in Washington. DSD has seen only fragments of the full series of records, sometimes in the form of single documents extracted from complete message packs. Of the records sighted by DSD, only two presented any problems, and the rest have been cleared for public release. The first, dating from 1944, contains details of decrypted Japanese cypher traffic. Its release is still being negotiated with the US Sigint authorities; DSD expects agreement to release the file will be forthcoming in the near future, and at that time it will be forwarded to Archives. It must be recognised that DSD cannot act unilaterally in releasing FRUMEL or Central Bureau records, since ownership of that material is shared with Australia's WWII allies.
[Note: This record has since been released and is available for public inspection.]
The other record is known as the FRUMEL 'Technical Record of Ciphers and Codes' (also referred to as the 'Red Book'). This was retained by DSD under a Determination issued in 1991 pursuant to Section 29 of the Archives Act (which enables certain records to be retained by DSD for reasons of national security or confidentiality) until early 1997, when it was passed to Australian Archives. At that time, certain sections of the document that described the technical details of Japanese cyphers were not released. The exemption was imposed because of a continuing, though low-level, risk that the material in question might compromise certain cryptanalytic capabilities. None of the exempted material contained any references or material relevant to the sinking of the Sydney. After consulting with US and UK Sigint authorities, DSD decided in mid-1997 to remove the exemption and make the document available for public access in its totality. The Determination, which was only one remaining in force in respect of wartime Sigint records, was formally revoked on 2 June.
It is to be noted that in this case DSD was prepared to release the exempted material, despite its potential to create a low-level risk to Sigint capabilities, solely in order to allay public concerns about the withholding of records relevant to the Sydney. The release of the record in its totality should make it clear that the earlier exemption of certain portions of the 'Red Book' until mid-1997, which was subject to some speculation, was in no way connected with any intention to suppress material that might cast light on the loss of the Sydney.
Records relating to HMAS Sydney
As will be apparent from Table 1, DSD has encountered only scattered references to the Sydney in all of the material so far submitted for inspection, none of which cast any light on its ultimate fate. None has any relationship to Sigint material. All of these records have been released for public access, and none, either in whole or part, is the subject of any exemption imposed at the request of DSD.
The FRUMEL 'Red Book' contained one reference to the Sydney. This was a brief account of the Unit's successful decryption of the material confiscated from the Captain of the Kormoran after his escape from custody and subsequent recapture. This extract from the 'Red Book' was released for public access in 1996, in recognition of the need to place all references to the Sydney on the public record.
Current exemptions
In November 1991, DSD requested a total of 218 excisions be made to a series of messages dating from 1 May 1940 to 31 December 1941 prior to their public release. The request was made at the behest of the UK Sigint authorities, under standard reciprocal arrangements, who indicated they wished certain material to be afforded continuing protection. The records in question comprise a small part of much larger series, and are designated MP1074/8, Box 1: 'Out' messages from ACNB. The complete MP1074 series occupies more than 300 linear metres of shelf space; DSD was requested to inspect only a portion of the whole. The material excised comprises the only WWII records currently subject to exemption under Section 33 of the Archives Act. None of the excisions relate to the Sydney. The records in question were forwarded to Australian Archives in 1991 and 1992. Since the UK's rules on disclosure of information have relaxed since the exemptions were imposed, DSD intends to seek agreement of the UK Sigint authorities for release of the material excised.
DSD understands that, for reasons that are not clear, these records cannot be located in the Australian Archives repository in Melbourne.
[Note: Subsequent to the Inquiry's March 1998 public hearing in Canberra, DSD archives personnel, in concert with staff from National Archives of Australia, located duplicate copies of all but three of the missing documents in NAA's Melbourne repository.]
To alleviate any concern on the part of the Inquiry as to the nature of the exemption, details of the excisions are as follows:
| REFERENCE EXCISED | ACTION |
|---|---|
| Secret designator of Allied entity | 197 occurrences of address excised from message address groups or message texts |
| Other sensitive message addresses | 9 excisions from message address groups or message texts |
| Operational information, mainly relating to Allied Sigint collaboration | 3 excisions from message texts |
| Information relating to cryptography | 5 exemptions of message texts |
| Names of personnel | 4 excisions from address groups or texts |
As the above table demonstrates, the vast majority of the excisions apply to telegraphic address designators (usually trigraphs or tetragraphs) that identify certain sensitive WWII offices. DSD expects to be in a position to lift the exemptions once the files have been located, and the UK authorities consulted.
[Note: All the above exemptions were lifted in June 1998 after DSD had completed an examination of the duplicate copies recovered in Melbourne.]
Records held by WWII allies
DSD has made enquiries of the agencies with whom Australia co-operated in Sigint activity during WWII. These agencies have indicated that their records contain no references to the loss of the Australian warship. In the case of the US, the records with the potential to contain relevant information are the FRUMEL and Central Bureau records, which, as indicated earlier, have been declassified and are open to public access. In the case of the UK, the relevant records would be those of the Far East Combined Bureau, which conducted Sigint operations from Hong Kong, and later from Singapore, in the early years of the War. These records were released by the UK authorities in April 1995.
In addition to the formal records of FRUMEL and Central Bureau, a number of historical accounts of the two units have been compiled by former serving personnel, including one issued by the US' Center for Cryptologic History. DSD has also two informal records of the units. DSD is prepared to make these documents available to Professor Dennis if they would assist him in researching any aspect of the early history of Australian Sigint that might have a bearing on the fate of the Sydney.
Another avenue of enquiry might lie in an examination of the records of the Dutch intelligence organisations. The Dutch, as administrators of the territory then known as the Dutch East Indies (now Indonesia), were actively engaged in Sigint against the Japanese in the early period of the War, and there was some collaboration between them and the British FECB. The RAN were also involved in Sigint activity with the Dutch. Informal research by DSD indicates that in the course of 1941 there were discussions between the Dutch and FECB in regard to joint activity against Japanese diplomatic traffic and in Direction Finding (DF). It is understood, however, that many of the Dutch records were destroyed as a consequence of the Japanese occupation of the Dutch East Indies, and also during the German occupation of the Netherlands.
As the Inquiry will be aware, DSD's records management staff have had a number of discussions with Professor Dennis in regard to uncovering further potential sources of information on the Sydney, and DSD will be more than happy to provide any further information that might be of assistance in identifying additional areas for research.
Summary
In summary, the Directorate advises the Inquiry that:
- DSD holds no records of its own relating to the loss of the Sydney,
other than administrative files generated recently in response to enquiries
from the public, or Questions in Parliament
- Of the records forwarded to DSD for examination by other elements
of Defence, or government Departments, in accordance with standing security
instructions for disposal of Sigint-related material, only a few records
were identified that contained references to the Sydney. None
are Sigint-related, and none have been the subject of an exemption pursuant
to section 33 of the Archives Act. All of these records are open to
public access. DSD has only one WWII file currently under examination,
and it is being processed for release. It also contains no reference
to the Sydney
- Exemptions from the 30-year rule are currently in place in respect
of a limited series of Naval records which contain Sigint information
dating from the early period of WWII. None of the records in question
contain references to the Sydney. The records are in the custody
of Australian Archives. DSD is prepared to lift the exemptions, subject
to agreement from the UK authorities, at whose behest the exemptions
were originally imposed in 1991
- DSD does not retain any WWII record under a Determination pursuant to section 29 of the Archives Act. The last remaining such Determination, which related to a technical document produced by FRUMEL, was revoked in June 1997. A reference in that document to the Sydney was released in 1996.
Conclusion
There is no material relating to the loss of the Sydney held or examined by DSD which is not publicly available. Over the past few years, DSD has made a strong effort to identify any documents that might have a bearing on the matter, with a view to facilitating their placement on the public record. On this basis, the Directorate can advise the Inquiry that existing Sigint records shed no light on the sea-battle between the Sydney and the Kormoran in November 1941, or the ultimate fate of the Australian warship.
Defence Signals Directorate
Canberra
March 1998