goals

[Mac]

Let me further amplify my thoughts on version compatibility.

The only time a user should encounter a compatibility issue is if data has been formatted at a higher level than the software supports. Suppose NewCrypt 7.0 supports some future AES-7 algorithm and a volume is formatted using that algorithm. But the user somehow tries to access the volume with an old copy, say NewCrypt 6.28, which does not implement AES-7.

Then NewCrypt 6.28 and the volume should have sufficient format protocol to gracefully identify and clearly diagnose this forward incompatibility.

However, conversely, even if we believe, say e.g., Serpent is no longer ideal and it is deprecated in release 7.0, then NewCrypt 7.0 should not refuse to access an existing volume formatted with Serpent. And therefore, as distasteful and cumbersome as it may seem, NewCrypt should continue to support Serpent indefinitely for compatibility's sake. Just as I assume the Linux kernel will continue to support ext3 and FAT32 for the indefinite future.

[Mac]

By my reference in the post of 09 Jun at 11:31am to the great complexity of the "system interface", I mean the many intricate subtle interfaces between the various file systems and the TrueCrypt I/O interceptions, manipulations and management. In comparison, the logic to do the actual cryptography is relatively minor. At least this is the case in Windows. I can't speak to Linux or Mac. Maybe they allow more streamlined interception.

[compul]

@Resonance, @Merlin, @compul
If you must deprecate a format or algorithm because it is has become notoriously broken, then you should, at worst, inhibit that format or algorithm being further used to create *fresh* instances -- but do not invalidate existing instances.

If I haven't made it clear enough in my former posts, I'll state it again here. For usability and maintaining the rich TC userbase, it is vital to have downward-compatibility for already existant volumes. There will be many users who do not follow the development for a long while, even years, and then install a new version. This will break their old containers and they will be rightfully pissed.

However I do consider it an important option to turn off "active" support (i.e. creation) for algorithms or formats that seem to have inherent weaknesses.

compul

[Resonance]

"comes across as rather arrogant and demeaning."
Sorry about that. I can re-word that, but some boards dislike edits. I don't know about here.
I think we all know an older person who barely can check their mail. You are correct. Easy for EVERYONE. No disrespect intended for anyone.

[Merlin]

@Resonance, @Merlin, @compul
If you must deprecate a format or algorithm because it is has become notoriously broken, then you should, at worst, inhibit that format or algorithm being further used to create *fresh* instances -- but do not invalidate existing instances.

If I haven't made it clear enough in my former posts, I'll state it again here. For usability and maintaining the rich TC userbase, it is vital to have downward-compatibility for already existant volumes. There will be many users who do not follow the development for a long while, even years, and then install a new version. This will break their old containers and they will be rightfully pissed.

However I do consider it an important option to turn off "active" support (i.e. creation) for algorithms or formats that seem to have inherent weaknesses.

compul

I'm with you on retaining compatibility with current volumes if that can be done while also moving forward on dealing with the audit issues, particularly the password hashing problems. I don't see any issue with keeping the current ciphers either, the defaults are sensible, anyone who doesn't understand why they'd switch from AES to twofish or serpent (or a cascade) tends to leave the defaults.

There are limits how long it's sensible to drag old code along after you, even microsoft are prepared to break backwards compatibility eventually, if you were to drop support for CBC mode volumes it would only affect volumes created by version 4.0 and earlier, CBC having been replaced in the first place owing to specific security concerns, namely vulnerability to watermarking attacks, people should no longer be using them, painful transition or not. These ancient creaky volume formats also lack the recovery options supported by later versions, specifically embedded backup headers.

The story is somewhat similar with the switch from LRW to XTS volume formats, it was made a very long time ago I can't recall the transition point but it's been years, and the reasons for it were security related, nagging people to please move on, this will be deprecated in future versions isn't unreasonable. Or likely to break things for that many users.

One of the difficulties with trying to support multiple password hashing functions, and these need modernising, as a direct result of the audit, even if only during a transition period, is that space for the boot loader code is limited, it simply may not be possible to make it fit and support both methods, in which case setup may have to perform checks and force a decrypt before any upgrade. I suspect backwards support can be managed if the hashing iteration is simply increased, trying two iteration counts is a trivial amount of extra code, changing the function to some other memory hard function is desirable, but where I'd expect the developers to simply be forced to introduce breakage, purely because they can't get a quart into a pint pot.

It's unfortunate, but in my personal opinion, dealing with the audit's security concerns really must take precedence over user convenience, any transition should be as seamless as possible, but eventually users who won't update need pushing into it for their own good. Setting such breakage points at the next major version with the prior warning them to update now, is about the least painless way I can think of to manage it.

I'm not certain how much help I can be to the team, I'm merely a long time user who's read the documentation thoroughly, no coder, nor crypto expert.

A note on the password length limitation, it may seem arbitrary that it's set to 64 characters, because of the way things are set up internally, passwords over this length add no extra security, it's explained in the documentation, I think the developers felt that it was better not to allow a false sense of security, than to have people thinking it was stronger than it was, I'm not sure how fixable it is without either introducing breakage, or introducing a load more legacy support to drag along and deprecate later.

[Resonance]

Something to consider:
Users who use CONTAINERS and users who use WDE are going to feel a completely different level of pain when things get changed/made better/deprecated/unencumbered/etc.
Somebody else made that point, and it's important, I think.

[Merlin]

Something to consider:
Users who use CONTAINERS and users who use WDE are going to feel a completely different level of pain when things get changed/made better/deprecated/unencumbered/etc.
Somebody else made that point, and it's important, I think.

Yes indeed, file based containers are going to present a very low bar to move forward, creating another and moving data to it isn't likely to be painful for most.
Partition based will present the most options, with the choice of decrypt / re-encrypt or backup recreate and restore.
WDE (especially system) will be most painful and hazardous.

Once you involve hidden volumes and/or hidden operating systems it could become nightmarish.

In all cases in-place migration would be nice to have, but I appreciate it may be either impossible or impose too much work for the developers.

[WaywardGeek]

Thanks for the detailed post. There's a lot there, and I've replied inline below.

@Resonance
1) You are discussing cutting down the number of algorithms.

I suggest this is not necessarily a useful notion.

Dropping support for some features would be useful in some ways, but is not necessarily the right way to move forward. In particular, it would improve security of the remaining code, since there would be less code to audit and maintain, and we could do a better job. However, I agree that this is just the sort of difference between a supported commercial tool and a hacker tool. I read in one post that TrueCrypt dropped support for various formats over time, but it's generally high backwards compatibility likely is one reason for its popularity.

I do not think we should drop encryption formats for the sake of maintaining less code. However, some features are harder to maintain than others. One of the questions I have not formed any opinion on is whether we should replace BIOS support with UEFI. The BIOS driver has so many issues that the audit had to list them in an appendix, and it requires using a Microsoft compiler from 1991. The audit team recommends upgrading the build tool chain, and I don't know how to keep BIOS support without the old one (does anyone know if it can be done?). However, I am not familiar with the issues involved, including whether or not dropping BIOS support would in any way impact current installations, or how many users lack UEFI. This should be discussed more, IMO.

@Resonance
The bulk of complexity you'll need to deal with is not in the crypto or the algorithms. You will find the system interfaces to be many binary orders of magnitude more complex than the crypto algorithms -- which are pretty simple, stable, self-contained and cut-and-dried.

From a user point of view, encryption algorithms Just Work. You call AES-Encrypt, and everything is dandy. You assume whoever is in charge of that code knows what they are doing. Unfortunately, we are those guys!

To create that impression, teams like this have to do an insane amount of work on the crypto, and even then, we will sometimes get it wrong! Even standard implementations of AES have been found to be insecure for crazy reasons, like when researchers were able to extract AES passwords by simply running their process on the same machine as AES. IIRC, they used a cache timing side-channel attack. Does TC still use an insecure implementation of AES? I haven't checked, but the audit surely will! For *dramatically* better performance, should we use the new Intel AES-NI instructions, or are we too afraid that there is somehow an NSA back door in there? Even AES file encryption needs random numbers, so should we use Intel's new RDRAND instruction to help generate the entropy? Remember the recent controversy over using RDRAND in the Linux entropy pool? If we pass on AES-NI, how about supporting the latest Haswell AVX2 instructions? These are a few of the outstanding issues for the most fundamental crypto in TC - AES! Writing and maintaining all that GUI code is simple in comparison. One reason I am very interested in working with the other FOSS competitors to TC, is that we may be able to share most of the basic crypto. For example, should we just link to the standard crypto library (-lcrypto)? These are critical design issues we need to start researching.

Now, assuming we get the core crypto right (God help us if we mess up AES), we still have lot of crypto to work on! It is not at all simple to turn a block cipher like AES into a secure, high performance encrypted volume format! Even the experts make mistakes. This is one reason why some formats in TC can only be read, and not created - some older TC formats were found to be insecure after years of use. The current TC volume format is being panned by some of the guys who maintain competing "better" formats (according to them - I am not yet qualified to judge). This is extremely difficult code to get right. This is why I feel strongly that the best crypto devs should have the final say over all code-related decisions, and if they say drop features that users still want, we should do it.

@Resonance
So eliminating algorithms is not really going to meaningfully reduce the complexity, or make a software audit significantly easier.

I'll let the cryptanalysts auditing the TC code know . I'm sure the vast number of algorithms supported in no way makes cryptanalysis harder

I mostly agree with the rest of your comments. I don't think anyone is seriously considering dropping support for a lot of previous encryption algorithms and volume formats. VeraCrypt did that, and to use it, you have to re-create your volume. This is one reason we're not begging to make the VeraCrypt fork the future of TrueCrypt. We obviously will not abandon Windows users, even if some developers on the project are anti-Windows. I personally am anti-Apple, but that doesn't mean we'll drop Mac support! Unless I have my way

[WaywardGeek]

Let me further amplify my thoughts on version compatibility.

The only time a user should encounter a compatibility issue is if data has been formatted at a higher level than the software supports. Suppose NewCrypt 7.0 supports some future AES-7 algorithm and a volume is formatted using that algorithm. But the user somehow tries to access the volume with an old copy, say NewCrypt 6.28, which does not implement AES-7.

Then NewCrypt 6.28 and the volume should have sufficient format protocol to gracefully identify and clearly diagnose this forward incompatibility.

If we can't decrypt the header, then there's nothing we can do, since TC volumes appear to be nothing but random data. So, for example, if we add a memory-hard password hashing algorithm, as I strongly believe we should, and a user tries to open a volume using that algorithm with an old copy of TC, it will fail and not be able to determine why. If we succeed in decrypting the header, then we could inform the user if our software is too out of data compared to the volume, or if his volume for any reason should upgrade it's format.

Let me further amplify my thoughts on version compatibility.
However, conversely, even if we believe, say e.g., Serpent is no longer ideal and it is deprecated in release 7.0, then NewCrypt 7.0 should not refuse to access an existing volume formatted with Serpent. And therefore, as distasteful and cumbersome as it may seem, NewCrypt should continue to support Serpent indefinitely for compatibility's sake. Just as I assume the Linux kernel will continue to support ext3 and FAT32 for the indefinite future.

I'm sure the team will take backwards compatibility seriously, and only drop support for a format or algorithm for compelling reasons after weighing the tradeoffs.

[Resonance]

One reason I am very interested in working with the other FOSS competitors to TC, is that we may be able to share most of the basic crypto. For example, should we just link to the standard crypto library (-lcrypto)? These are critical design issues we need to start researching."

That's an interesting idea. Maybe make an agreement to write and audit basic crypto modules together (and use the same string names, etc.) with the other FOSS competitors to TC. You would make different cars, but you can agree on one carburetor, how it bolts on, etc.

When it comes to WDE, BIOS, perhaps headers, GUI, collecting entropy in more interesting ways, splash screens, and labelling, the different teams could politely part ways.