290 lines
11 KiB
HTML
290 lines
11 KiB
HTML
<!DOCTYPE html>
|
|
<html><head>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
|
|
<link href="sqlite.css" rel="stylesheet">
|
|
<title>The Checksum VFS Shim</title>
|
|
<!-- path= -->
|
|
</head>
|
|
<body>
|
|
<div class=nosearch>
|
|
<a href="index.html">
|
|
<img class="logo" src="images/sqlite370_banner.gif" alt="SQLite" border="0">
|
|
</a>
|
|
<div><!-- IE hack to prevent disappearing logo --></div>
|
|
<div class="tagline desktoponly">
|
|
Small. Fast. Reliable.<br>Choose any three.
|
|
</div>
|
|
<div class="menu mainmenu">
|
|
<ul>
|
|
<li><a href="index.html">Home</a>
|
|
<li class='mobileonly'><a href="javascript:void(0)" onclick='toggle_div("submenu")'>Menu</a>
|
|
<li class='wideonly'><a href='about.html'>About</a>
|
|
<li class='desktoponly'><a href="docs.html">Documentation</a>
|
|
<li class='desktoponly'><a href="download.html">Download</a>
|
|
<li class='wideonly'><a href='copyright.html'>License</a>
|
|
<li class='desktoponly'><a href="support.html">Support</a>
|
|
<li class='desktoponly'><a href="prosupport.html">Purchase</a>
|
|
<li class='search' id='search_menubutton'>
|
|
<a href="javascript:void(0)" onclick='toggle_search()'>Search</a>
|
|
</ul>
|
|
</div>
|
|
<div class="menu submenu" id="submenu">
|
|
<ul>
|
|
<li><a href='about.html'>About</a>
|
|
<li><a href='docs.html'>Documentation</a>
|
|
<li><a href='download.html'>Download</a>
|
|
<li><a href='support.html'>Support</a>
|
|
<li><a href='prosupport.html'>Purchase</a>
|
|
</ul>
|
|
</div>
|
|
<div class="searchmenu" id="searchmenu">
|
|
<form method="GET" action="search">
|
|
<select name="s" id="searchtype">
|
|
<option value="d">Search Documentation</option>
|
|
<option value="c">Search Changelog</option>
|
|
</select>
|
|
<input type="text" name="q" id="searchbox" value="">
|
|
<input type="submit" value="Go">
|
|
</form>
|
|
</div>
|
|
</div>
|
|
<script>
|
|
function toggle_div(nm) {
|
|
var w = document.getElementById(nm);
|
|
if( w.style.display=="block" ){
|
|
w.style.display = "none";
|
|
}else{
|
|
w.style.display = "block";
|
|
}
|
|
}
|
|
function toggle_search() {
|
|
var w = document.getElementById("searchmenu");
|
|
if( w.style.display=="block" ){
|
|
w.style.display = "none";
|
|
} else {
|
|
w.style.display = "block";
|
|
setTimeout(function(){
|
|
document.getElementById("searchbox").focus()
|
|
}, 30);
|
|
}
|
|
}
|
|
function div_off(nm){document.getElementById(nm).style.display="none";}
|
|
window.onbeforeunload = function(e){div_off("submenu");}
|
|
/* Disable the Search feature if we are not operating from CGI, since */
|
|
/* Search is accomplished using CGI and will not work without it. */
|
|
if( !location.origin || !location.origin.match || !location.origin.match(/http/) ){
|
|
document.getElementById("search_menubutton").style.display = "none";
|
|
}
|
|
/* Used by the Hide/Show button beside syntax diagrams, to toggle the */
|
|
function hideorshow(btn,obj){
|
|
var x = document.getElementById(obj);
|
|
var b = document.getElementById(btn);
|
|
if( x.style.display!='none' ){
|
|
x.style.display = 'none';
|
|
b.innerHTML='show';
|
|
}else{
|
|
x.style.display = '';
|
|
b.innerHTML='hide';
|
|
}
|
|
return false;
|
|
}
|
|
</script>
|
|
</div>
|
|
<div class=fancy>
|
|
<div class=nosearch>
|
|
<div class="fancy_title">
|
|
The Checksum VFS Shim
|
|
</div>
|
|
<div class="fancy_toc">
|
|
<a onclick="toggle_toc()">
|
|
<span class="fancy_toc_mark" id="toc_mk">►</span>
|
|
Table Of Contents
|
|
</a>
|
|
<div id="toc_sub"><div class="fancy-toc1"><a href="#overview">1. Overview</a></div>
|
|
<div class="fancy-toc1"><a href="#compiling">2. Compiling</a></div>
|
|
<div class="fancy-toc1"><a href="#loading">3. Loading</a></div>
|
|
<div class="fancy-toc1"><a href="#usage">4. Usage</a></div>
|
|
<div class="fancy-toc1"><a href="#verification_of_checksums">5. Verification Of Checksums</a></div>
|
|
<div class="fancy-toc1"><a href="#controlling_checksum_verification">6. Controlling Checksum Verification</a></div>
|
|
</div>
|
|
</div>
|
|
<script>
|
|
function toggle_toc(){
|
|
var sub = document.getElementById("toc_sub")
|
|
var mk = document.getElementById("toc_mk")
|
|
if( sub.style.display!="block" ){
|
|
sub.style.display = "block";
|
|
mk.innerHTML = "▼";
|
|
} else {
|
|
sub.style.display = "none";
|
|
mk.innerHTML = "►";
|
|
}
|
|
}
|
|
</script>
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
<h1 id="overview"><span>1. </span>Overview</h1>
|
|
|
|
<p>The checksum VFS extension is a <a href="vfs.html#shim">VFS shim</a> that adds an 8-byte
|
|
checksum to the end of every page in an SQLite database. The checksum
|
|
is added as each page is written and verified as each page is read.
|
|
The checksum is intended to help detect database corruption caused by
|
|
random bit-flips in the mass storage device.
|
|
|
|
</p><p>The checksum VFS extension requires
|
|
SQLite version 3.32.0 (2020-05-22) or later. It will not
|
|
work with earlier versions of SQLite.
|
|
|
|
</p><h1 id="compiling"><span>2. </span>Compiling</h1>
|
|
|
|
<p>The checksum VFS module is a <a href="loadext.html">loadable extension</a>. It is not
|
|
included in the <a href="amalgamation.html">amalgamation</a>. It must be added to SQLite
|
|
either at compile-time or at run-time. The source code to
|
|
the checksum VFS module is in the
|
|
<a href="https://sqlite.org/src/file/ext/misc/cksumvfs.c">ext/misc/cksumvfs.c</a>
|
|
source file in the
|
|
<a href="https://sqlite.org/src">SQLite source tree</a>.
|
|
|
|
</p><p>To build the checksum VFS module into a run-time loadable
|
|
extension, use commands similar to the following:
|
|
|
|
</p><ul>
|
|
<li>(linux) → <tt>gcc -fPIC -shared cksumvfs.c -o cksumvfs.so</tt>
|
|
</li><li>(mac) → <tt>clang -fPIC -dynamiclib cksumvfs.c -o cksumvfs.dylib</tt>
|
|
</li><li>(windows) → <tt>cl cksumvfs.c -link -dll -out:cksumvfs.dll</tt>
|
|
</li></ul>
|
|
|
|
<p>You may want to add additional compiler options, of course,
|
|
according to the needs of your project.
|
|
|
|
</p><p>To statically link this extension with your product,
|
|
compile it like any other C-language module but add the
|
|
"-DSQLITE_CKSUMVFS_STATIC" option so that this module knows that
|
|
it is being statically linked rather than dynamically linked.
|
|
|
|
</p><h1 id="loading"><span>3. </span>Loading</h1>
|
|
|
|
<p>To load this extension as a shared library, you first have to
|
|
bring up a dummy SQLite database connection to use as the argument
|
|
to the <a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API call. Then you invoke the
|
|
<a href="c3ref/load_extension.html">sqlite3_load_extension()</a> API and shutdown the dummy database
|
|
connection. All subsequent database connections that are opened
|
|
will include this extension. For example:
|
|
|
|
</p><div class="codeblock"><pre>sqlite3 *db;
|
|
sqlite3_open(":memory:", &db);
|
|
sqlite3_load_extension(db, "./cksumvfs");
|
|
sqlite3_close(db);
|
|
</pre></div>
|
|
|
|
<p>If this extension is compiled with -DSQLITE_CKSUMVFS_STATIC and
|
|
statically linked against the application, initialize it using
|
|
a single API call as follows:
|
|
|
|
</p><div class="codeblock"><pre>sqlite3_cksumvfs_init();
|
|
</pre></div>
|
|
|
|
<p>Cksumvfs is a <a href="vfs.html#shim">VFS shim</a>. When loaded, "cksmvfs" becomes the new
|
|
default VFS, and it uses the prior default VFS as the next VFS
|
|
down in the stack. This is normally what you want. However, in
|
|
complex situations where multiple VFS shims are being loaded,
|
|
it might be important to ensure that cksumvfs is loaded in the
|
|
correct order so that it sequences itself into the default VFS
|
|
Shim stack in the right order.
|
|
|
|
</p><h1 id="usage"><span>4. </span>Usage</h1>
|
|
|
|
<p>Open database connections using the <a href="c3ref/open.html">sqlite3_open()</a> or
|
|
<a href="c3ref/open.html">sqlite3_open_v2()</a> interfaces, as normal. Ordinary database files
|
|
(without a checksum) will operate normally. Databases with
|
|
checksums will return an SQLITE_IOERR_DATA error if a page is
|
|
encountered that contains an invalid checksum.
|
|
|
|
</p><p>Checksumming only works on databases that have a <a href="fileformat2.html#resbyte">reserve bytes</a>
|
|
value of exactly 8. The default value for reserve-bytes is 0.
|
|
Hence, newly created database files will omit the checksum by
|
|
default. To create a database that includes a checksum, change
|
|
the reserve-bytes value to 8 by running code similar to this:
|
|
|
|
</p><div class="codeblock"><pre>int n = 8;
|
|
sqlite3_file_control(db, 0, SQLITE_FCNTL_RESERVED_BYTES, &n);
|
|
</pre></div>
|
|
|
|
<p>If you do this immediately after creating a new database file,
|
|
before anything else has been written into the file, then that
|
|
might be all that you need to do. Otherwise, the API call
|
|
above should be followed by:
|
|
|
|
</p><div class="codeblock"><pre>sqlite3_exec(db, "VACUUM", 0, 0, 0);
|
|
</pre></div>
|
|
|
|
<p>It never hurts to run the VACUUM, even if you don't need it.
|
|
If the database is in WAL mode, you should shutdown and
|
|
reopen all database connections before continuing.
|
|
|
|
</p><p>From the CLI, use the ".filectrl reserve_bytes 8" command,
|
|
followed by "VACUUM;".
|
|
|
|
</p><p>Note that SQLite allows the number of reserve-bytes to be
|
|
increased but not decreased. So if a database file already
|
|
has a reserve-bytes value greater than 8, there is no way to
|
|
activate checksumming on that database, other than to dump
|
|
and restore the database file. Note also that other extensions
|
|
might also make use of the reserve-bytes. Checksumming will
|
|
be incompatible with those other extensions.
|
|
|
|
</p><h1 id="verification_of_checksums"><span>5. </span>Verification Of Checksums</h1>
|
|
|
|
<p>If any checksum is incorrect, the "PRAGMA quick_check" command
|
|
will find it. To verify that checksums are actually enabled
|
|
and running, use SQL like the following:
|
|
|
|
</p><div class="codeblock"><pre>SELECT count(*), verify_checksum(data)
|
|
FROM sqlite_dbpage
|
|
GROUP BY 2;
|
|
</pre></div>
|
|
|
|
<p>There are three possible outputs from the verify_checksum()
|
|
function: 1, 0, and NULL. 1 is returned if the checksum is
|
|
correct. 0 is returned if the checksum is incorrect. NULL
|
|
is returned if the page is unreadable. If checksumming is
|
|
enabled, the read will fail if the checksum is wrong, so the
|
|
usual result from verify_checksum() on a bad checksum is NULL.
|
|
|
|
</p><p>If everything is OK, the query above should return a single
|
|
row where the second column is 1. Any other result indicates
|
|
either that there is a checksum error, or checksum validation
|
|
is disabled.
|
|
|
|
</p><h1 id="controlling_checksum_verification"><span>6. </span>Controlling Checksum Verification</h1>
|
|
|
|
<p>The cksumvfs extension implements a new PRAGMA statement that can
|
|
be used to disable, re-enable, or query the status of checksum
|
|
verification:
|
|
|
|
</p><div class="codeblock"><pre>PRAGMA checksum_verification; -- query status
|
|
PRAGMA checksum_verification=OFF; -- disable verification
|
|
PRAGMA checksum_verification=ON; -- re-enable verification
|
|
</pre></div>
|
|
|
|
<p>The "checksum_verification" pragma will return "1" (true) or "0"
|
|
(false) if checksum verification is enabled or disabled, respectively.
|
|
"Verification" in this context means the feature that causes
|
|
SQLITE_IOERR_DATA errors if a checksum mismatch is detected while
|
|
reading. Checksums are always kept up-to-date as long as the
|
|
<a href="fileformat2.html#resbyte">reserve bytes</a> value of the database is 8, regardless of the setting
|
|
of this pragma. Checksum verification can be disabled (for example)
|
|
to do forensic analysis of a database that has previously reported
|
|
a checksum error.
|
|
|
|
</p><p>The "checksum_verification" pragma will always respond with "0" if
|
|
the database file does not have a <a href="fileformat2.html#resbyte">reserve bytes</a> value of 8. The
|
|
pragma will return no rows at all if the cksumvfs extension is
|
|
not loaded.
|
|
</p>
|