89 Commits

Author SHA1 Message Date
Nazar Kanaev
3ab098db5c update changelog 2023-08-21 10:39:19 +01:00
Nazar Kanaev
6d16e93008 fix sqlite conflict handling for feeds/folders 2023-08-19 21:51:05 +01:00
Nazar Kanaev
98934daee4 update docs 2023-08-15 14:35:42 +01:00
Nazar Kanaev
259474cae9 v2.4 2023-08-15 14:13:12 +01:00
Nazar Kanaev
1e65a7951b ui fixes 2023-08-14 22:01:13 +01:00
Nazar Kanaev
bed5640366 fix base font size 2023-08-14 21:25:47 +01:00
Hannes Braun
57ea83cf4f Use more recent images for GitHub Actions 2023-08-13 22:29:41 +01:00
Hannes Braun
219842d723 Update minimum required Go version to 1.17 2023-08-13 22:29:41 +01:00
Nazar Kanaev
a96fc101f2 paragraph margins 2023-08-06 12:40:54 +01:00
Nazar Kanaev
81a77ce0a4 update changelog 2023-05-22 22:28:15 +01:00
Adam Szkoda
9ed359f964 Protect a read from a map with a mutex 2023-05-22 22:24:57 +01:00
Nazar Kanaev
bc18557820 handle isPermalink in rss feeds 2023-05-20 23:26:22 +01:00
Nazar Kanaev
7d99edab8d update changelog 2023-05-16 11:12:54 +01:00
x2cf
32ca121520 fix item order 2023-05-16 11:09:57 +01:00
nkanaev
9f1a0534a3 Update readme.md 2023-04-18 14:59:39 +01:00
Nazar Kanaev
d2678be96d update dependencies 2023-03-18 20:48:10 +00:00
nkanaev
95ebbb9d13 update changelog 2023-02-10 09:41:50 +00:00
Kaloyan Petrov
0f6d4d639d Firefox uses altKey to switch tabs 2023-02-10 09:40:03 +00:00
Kaloyan Petrov
795a5d2cb4 Use JS event.code to support different language layouts 2023-02-10 09:39:45 +00:00
Nazar Kanaev
dd5f760606 update docs 2023-02-09 21:37:56 +00:00
Nazar Kanaev
58d6a46e36 increase hidden input height
this allows to show the whole selectgroup area
when scrolled via arrow keys.

as a side effect it makes pagination to work with arrow keys as well.
2023-02-09 21:34:23 +00:00
Nazar Kanaev
a8d7b86cdc do not render root folder input 2023-02-06 11:37:16 +00:00
Nazar Kanaev
aac3de7ca2 make active toolbar buttons prominent 2023-02-02 22:40:41 +00:00
Nazar Kanaev
de24659bae update changelog 2023-02-02 22:09:13 +00:00
Nazar Kanaev
632412c10e go fmt 2023-02-02 22:06:27 +00:00
Nazar Kanaev
012b58bbe4 make newly added feed searchable 2023-02-02 22:05:58 +00:00
nkanaev
c092842ee4 Update changelog.txt 2023-01-30 22:11:00 +00:00
Aaron Bieber
e4c1d01915 Add manifest.json for better mobile integration
manifest.json allows yarr to run in a more app-like mode on mobile
devices.

More info here: https://developer.mozilla.org/en-US/docs/Web/Manifest
2023-01-30 22:05:33 +00:00
Nazar Kanaev
ce07ddea92 update changelog 2023-01-29 14:30:36 +00:00
Nazar Kanaev
bd6322e533 set article view width limit 2023-01-29 14:28:43 +00:00
nkanaev
91da774286 Create todo.txt 2022-11-16 10:21:55 +00:00
nkanaev
e62906e63d fix readability edge case 2022-11-14 15:11:05 +00:00
nkanaev
56e5625adc Update readme.md 2022-09-27 19:53:29 +01:00
Jake Bauer
1ecf4b0bb4 fix linux download instruction
The instruction for downloading yarr for linux told the user to download yar-*-windows which is incorrect. The Linux package on the releases page is yarr-*-linux64.zip.
2022-09-27 19:42:08 +01:00
Nazar Kanaev
57d9421c7f update readme.md 2022-09-09 22:16:41 +01:00
nkanaev
a73188944d Update readme.md 2022-09-09 22:14:37 +01:00
Nazar Kanaev
97904cc0f3 linux installation script 2022-09-09 22:13:55 +01:00
Nazar Kanaev
f28f354992 update changelog 2022-08-21 13:32:18 +01:00
Nazar Kanaev
698f5d6d06 handle google url redirect in page crawler 2022-08-21 13:31:18 +01:00
nkanaev
b935a1c511 Update changelog.txt 2022-08-01 23:20:09 +01:00
Pierre Prinetti
10e6bfa5a0 Set auth without authfile
With this patch, it is possible to pass username and password directly
on the command line with:

```shell
yarr --auth 'username:password'
```

The corresponding environment variable comes handy with containers:
```shell
YARR_AUTH='username:password' yarr
```
2022-07-06 15:21:41 +01:00
nkanaev
f030a4075b refactoring 2022-07-04 16:03:50 +01:00
Pierre Prinetti
c9dd977600 Allow passwords with column
Before this patch, an `authfile` with multiple column symbols was not
valid.

After this patch, all characters after the first `:` constitute the
password, until EOL.
2022-07-04 15:21:09 +01:00
Pierre Prinetti
c1bcc0c517 Run go fmt
This patch is the result of running `go fmt ./...` with Go v1.16.15.
2022-07-04 15:20:49 +01:00
nkanaev
2a5692d9a7 fix scrolling issue with large font 2022-06-29 11:41:01 +01:00
quoing
a8d160f9b1 Fix concurrent map writes for FeedIcon cache 2022-06-21 16:12:59 +01:00
nkanaev
286cbff236 update favicon (2nd attempt) 2022-06-06 10:04:47 +01:00
nkanaev
fff0870d3b update changelog 2022-06-02 10:18:17 +01:00
nkanaev
fe22460c07 favicon tweaks 2022-06-02 10:16:56 +01:00
Nazar Kanaev
18f2789a5d oopsies 2022-06-01 21:04:42 +01:00
Nazar Kanaev
7f161a5408 update changelog 2022-06-01 21:02:13 +01:00
Nazar Kanaev
cba3fbc48c fix append (different behaviour in arm) 2022-06-01 20:51:00 +01:00
Nazar Kanaev
5e46f1480e add vendor 2022-06-01 20:49:50 +01:00
Nazar Kanaev
ead253c55f cross-compilation for ARMv7 2022-06-01 20:49:38 +01:00
Nazar Kanaev
6b8da92cb3 update link 2022-06-01 13:26:03 +01:00
Nazar Kanaev
a91f64ce9d docker build instructions 2022-06-01 13:24:39 +01:00
Nazar Kanaev
e1a6ccf133 update changelog 2022-05-03 20:59:24 +01:00
Nazar Kanaev
d2c034a850 v2.3 2022-05-03 20:40:39 +01:00
Nazar Kanaev
713930decc update changelog 2022-05-03 15:45:22 +01:00
Nazar Kanaev
ee2a825cf0 get rss link when atom link is present
found in: https://rss.nytimes.com/services/xml/rss/nyt/Arts.xml

when both rss and atom link elements are present, xml parser returns
empty string. provide default namespace to capture rss link properly.
2022-05-03 15:35:57 +01:00
Nazar Kanaev
8e9da86f83 nah 2022-04-09 16:06:19 +01:00
Nazar Kanaev
9eb49fd3a7 credits 2022-04-09 16:03:46 +01:00
Nazar Kanaev
684bc25b83 fix: load more items to prevent scroll lock 2022-04-09 15:58:33 +01:00
nkanaev
8ceab03cd7 fix text color in dark mode 2022-03-01 10:43:25 +00:00
Nazar Kanaev
34dad4ac8f systray: fix build flag 2022-02-16 14:04:04 +00:00
Nazar Kanaev
b40d930f8a credits 2022-02-15 22:12:32 +00:00
Nazar Kanaev
d4b34e900e update test 2022-02-15 22:04:16 +00:00
Nazar Kanaev
954b549029 update 2022-02-15 21:56:32 +00:00
Nazar Kanaev
fbd0b2310e update changelog 2022-02-14 20:33:28 +00:00
Nazar Kanaev
be7af0ccaf handle invalid chars in non-utf8 xml 2022-02-14 15:23:55 +00:00
Nazar Kanaev
18221ef12d use bytes.Buffer instead 2022-02-14 11:05:38 +00:00
Nazar Kanaev
4c0726412b do not build systray in linux 2022-02-14 00:56:03 +00:00
Nazar Kanaev
d7253a60b8 strip out invalid xml characters 2022-02-12 23:42:44 +00:00
Nazar Kanaev
2de3ddff08 fix test 2022-02-12 23:41:01 +00:00
Nazar Kanaev
830248b6ae store feed size 2022-02-10 22:14:47 +00:00
Nazar Kanaev
f8db2ef7ad delete old items based on feed size 2022-02-10 22:14:47 +00:00
Nazar Kanaev
109caaa889 cascade 2022-02-10 22:14:47 +00:00
Nazar Kanaev
d0b83babd2 initial work for smarter database cleanup 2022-02-10 22:14:47 +00:00
nkanaev
de3decbffd remove unused assets 2022-01-26 10:44:33 +00:00
nkanaev
c92229a698 update changelog 2022-01-24 16:56:55 +00:00
nkanaev
176852b662 credits 2022-01-24 16:52:29 +00:00
nkanaev
52cc8ecbbd fix encoding 2022-01-24 16:47:32 +00:00
nkanaev
e3e9542f1e fix page crawling encoding 2022-01-24 14:02:21 +00:00
nkanaev
b78c8bf8bf fix parsing opml with encoding 2022-01-24 13:10:30 +00:00
nkanaev
bff7476b58 refactoring 2022-01-24 12:50:52 +00:00
Nazar Kanaev
05f5785660 update promo.png 2022-01-18 14:36:05 +00:00
David Adi Nugroho
cb50aed89a add placeholder and autofocus to new feed url 2021-12-28 17:56:26 +00:00
Nazar Kanaev
df655aca5e remove todo 2021-11-20 22:42:33 +00:00
Nazar Kanaev
86853a87bf update changelog 2021-11-20 22:07:33 +00:00
215 changed files with 453097 additions and 257 deletions

View File

@@ -7,7 +7,7 @@ on:
jobs:
build_macos:
name: Build for MacOS
runs-on: macos-10.15
runs-on: macos-13
steps:
- name: "Checkout"
uses: actions/checkout@v2
@@ -16,7 +16,7 @@ jobs:
- name: "Setup Go"
uses: actions/setup-go@v2
with:
go-version: '^1.16'
go-version: '^1.17'
- name: Cache Go Modules
uses: actions/cache@v2
with:
@@ -34,7 +34,7 @@ jobs:
build_windows:
name: Build for Windows
runs-on: windows-2019
runs-on: windows-2022
steps:
- name: "Checkout"
uses: actions/checkout@v2
@@ -43,7 +43,7 @@ jobs:
- name: "Setup Go"
uses: actions/setup-go@v2
with:
go-version: '^1.16'
go-version: '^1.17'
- name: Cache Go Modules
uses: actions/cache@v2
with:
@@ -61,7 +61,7 @@ jobs:
build_linux:
name: Build for Linux
runs-on: ubuntu-18.04
runs-on: ubuntu-22.04
steps:
- name: "Checkout"
uses: actions/checkout@v2
@@ -70,7 +70,7 @@ jobs:
- name: "Setup Go"
uses: actions/setup-go@v2
with:
go-version: '^1.16'
go-version: '^1.17'
- name: Cache Go Modules
uses: actions/cache@v2
with:

44
build.md Normal file
View File

@@ -0,0 +1,44 @@
## Compilation
Install `Go >= 1.17` and `GCC`. Get the source code:
git clone https://github.com/nkanaev/yarr.git
Then run one of the corresponding commands:
# create an executable for the host os
make build_macos # -> _output/macos/yarr.app
make build_linux # -> _output/linux/yarr
make build_windows # -> _output/windows/yarr.exe
# host-specific cli version (no gui)
make build_default # -> _output/yarr
# ... or start a dev server locally
make serve # starts a server at http://localhost:7070
# ... or build a docker image
docker build -t yarr .
## ARM compilation
The instructions below are to cross-compile *yarr* to `Linux/ARM*`.
Build:
docker build -t yarr.arm -f dockerfile.arm .
Test:
# inside host
docker run -it --rm yarr.arm
# then, inside container
cd /root/out
qemu-aarch64 -L /usr/aarch64-linux-gnu/ yarr.arm64
Extract files from images:
CID=$(docker create yarr.arm)
docker cp -a "$CID:/root/out" .
docker rm "$CID"

View File

@@ -1,4 +1,36 @@
# upcoming
# upcoming
- (fix) duplicate articles caused by the same feed addition (thanks to @adaszko)
# v2.4 (2023-08-15)
- (new) ARM build support (thanks to @tillcash & @fenuks)
- (new) auth configuration via param or env variable (thanks to @pierreprinetti)
- (new) web app manifest for an app-like experience on mobile (thanks to @qbit)
- (fix) concurrency issue crashing the app (thanks to @quoing)
- (fix) favicon visibility in dark mode (thanks to @caycaycarly for the report)
- (fix) autoloading more articles not working in certain edge cases (thanks to @fenuks for the report)
- (fix) handle Google URL redirects in "Read Here" (thanks to @cubbei for discovery)
- (fix) handle failures to extract content in "Read Here" (thanks to @grigio for the report)
- (fix) article view width for high resolution screens (thanks to @whaler-ragweed for the report)
- (fix) make newly added feed searchable (thanks to @BMorearty for the report)
- (fix) feed/article selection accessibility via arrow keys (thanks to @grigio and @tillcash)
- (fix) keyboard shortcuts in Firefox (thanks to @kaloyan13)
- (fix) keyboard shortcuts in non-English layouts (thanks to @kaloyan13)
- (fix) sorting articles with timezone information (thanks to @x2cf)
- (fix) handling links set in guid only for certain feeds (thanks to @adaszko for the report)
- (fix) crashes caused by feed icon endpoint (thanks to @adaszko)
# v2.3 (2022-05-03)
- (fix) handling encodings (thanks to @f100024 & @fserb)
- (fix) parsing xml feeds with illegal characters (thanks to @stepelu for the report)
- (fix) old articles reappearing as unread (thanks to @adaszko for the report)
- (fix) item list scrolling issue on large screens (thanks to @bielej for the report)
- (fix) keyboard shortcuts color in dark mode (thanks to @John09f9 for the report)
- (etc) autofocus when adding a new feed (thanks to @lakuapik)
# v2.2 (2021-11-20)
- (fix) windows console support (thanks to @dufferzafar for the report)
- (fix) remove html tags from article titles (thanks to Alex Went for the report)

View File

@@ -1 +0,0 @@
- feedlist keyboard navigation is flaky in "unread" section

44
dockerfile.arm Normal file
View File

@@ -0,0 +1,44 @@
FROM ubuntu:20.04
# Install GCC
RUN apt update
RUN apt install -y \
wget build-essential \
gcc-aarch64-linux-gnu \
binutils-aarch64-linux-gnu binutils-aarch64-linux-gnu-dbg \
gcc-arm-linux-gnueabihf \
binutils-arm-linux-gnueabihf binutils-arm-linux-gnueabihf-dbg
RUN env DEBIAN_FRONTEND=noninteractive \
apt install -y qemu-user qemu-user-static
# Install Golang
RUN wget --quiet https://go.dev/dl/go1.18.2.linux-amd64.tar.gz && \
rm -rf /usr/local/go && \
tar -C /usr/local -xzf go1.18.2.linux-amd64.tar.gz
ENV PATH=$PATH:/usr/local/go/bin
# Copy source code
WORKDIR /root/src
RUN mkdir /root/out
COPY . .
# Build ARM64
RUN env \
CC=aarch64-linux-gnu-gcc \
CGO_ENABLED=1 \
GOOS=linux GOARCH=arm64 \
go build \
-tags "sqlite_foreign_keys release linux" \
-ldflags="-s -w" \
-o /root/out/yarr.arm64 src/main.go
RUN env \
CC=arm-linux-gnueabihf-gcc \
CGO_ENABLED=1 \
GOOS=linux GOARCH=arm GOARM=7 \
go build \
-tags "sqlite_foreign_keys release linux" \
-ldflags="-s -w" \
-o /root/out/yarr.arm7 src/main.go
CMD ["/bin/bash"]

23
etc/install-linux.sh Normal file
View File

@@ -0,0 +1,23 @@
#!/bin/bash
cat >"$HOME/.local/share/applications/yarr.desktop" <<END
[Desktop Entry]
Name=yarr
Exec=$HOME/.local/bin/yarr -open
Icon=yarr
Type=Application
Categories=Internet;
END
cat >"$HOME/.local/share/icons/yarr.svg" <<END
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-anchor-favicon">
<circle cx="12" cy="5" r="3" stroke-width="4" stroke="#ffffff"></circle>
<line x1="12" y1="22" x2="12" y2="8" stroke-width="4" stroke="#ffffff"></line>
<path d="M5 12H2a10 10 0 0 0 20 0h-3" stroke-width="4" stroke="#ffffff"></path>
<circle cx="12" cy="5" r="3"></circle>
<line x1="12" y1="22" x2="12" y2="8"></line>
<path d="M5 12H2a10 10 0 0 0 20 0h-3"></path>
</svg>
END

Binary file not shown.

Before

Width:  |  Height:  |  Size: 430 KiB

After

Width:  |  Height:  |  Size: 173 KiB

8
go.mod
View File

@@ -1,9 +1,11 @@
module github.com/nkanaev/yarr
go 1.16
go 1.17
require (
github.com/mattn/go-sqlite3 v1.14.7
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420
golang.org/x/sys v0.0.0-20210503173754-0981d6026fa6
golang.org/x/net v0.8.0
golang.org/x/sys v0.6.0
)
require golang.org/x/text v0.8.0 // indirect

41
go.sum
View File

@@ -1,12 +1,39 @@
github.com/mattn/go-sqlite3 v1.14.7 h1:fxWBnXkxfM6sRiuH3bqJ4CfzZojMOLVc0UTsTglEghA=
github.com/mattn/go-sqlite3 v1.14.7/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420 h1:a8jGStKg0XqKDlKqjLrXn0ioF5MH36pT7Z0BRTqLhbk=
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0 h1:Zrh2ngAOFYneWTAIAPethzeaQLuHwhuBkuV6ZiRnUaQ=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210503173754-0981d6026fa6 h1:cdsMqa2nXzqlgs183pHxtvoVwU7CyzaCTAUOg94af4c=
golang.org/x/sys v0.0.0-20210503173754-0981d6026fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0 h1:MVltZSvRTcU2ljQOhs94SXPftV6DCNnZViHeQps87pQ=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.8.0 h1:57P1ETyNKtuIjB4SRd15iJxuhj8Gc416Y78H3qgMh68=
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=

View File

@@ -1,4 +1,4 @@
VERSION=2.2
VERSION=2.4
GITHASH=$(shell git rev-parse --short=8 HEAD)
CGO_ENABLED=1

View File

@@ -3,67 +3,32 @@
**yarr** (yet another rss reader) is a web-based feed aggregator which can be used both
as a desktop application and a personal self-hosted server.
It is written in Go with the frontend in Vue.js. The storage is backed by SQLite.
The app is a single binary with an embedded database (SQLite).
![screenshot](etc/promo.png)
## usage
The latest prebuilt binaries for Linux/MacOS/Windows are available
The latest prebuilt binaries for Linux/MacOS/Windows AMD64 are available
[here](https://github.com/nkanaev/yarr/releases/latest).
### macos
Download `yarr-*-macos64.zip`, unzip it, place `yarr.app` in `/Applications` folder.
To open the app follow the instructions provided [here][macos-open] or run the command below:
xattr -d com.apple.quarantine /Applications/yarr.app
Download `yarr-*-macos64.zip`, unzip it, place `yarr.app` in `/Applications` folder, [open the app][macos-open], click the anchor menu bar icon, select "Open".
[macos-open]: https://support.apple.com/en-gb/guide/mac-help/mh40616/mac
### windows
Download `yarr-*-windows32.zip`, unzip it, open `yarr.exe`
Download `yarr-*-windows64.zip`, unzip it, open `yarr.exe`, click the anchor system tray icon, select "Open".
### linux
The Linux version doesn't come with the desktop environment integration.
For easy access on DE it is recommended to create a desktop menu entry by
by following the steps below:
unzip -x yarr*.zip
sudo mv yarr /usr/local/bin/yarr
sudo nano /usr/local/share/applications/yarr.desktop
and pasting the content:
[Desktop Entry]
Name=yarr
Exec=/usr/local/bin/yarr -open
Icon=rss
Type=Application
Categories=Internet;
Download `yarr-*-linux64.zip`, unzip it, place `yarr` in `$HOME/.local/bin`
and run [the script](etc/install-linux.sh).
For self-hosting, see `yarr -h` for auth, tls & server configuration flags.
## build
Install `Go >= 1.16` and `gcc`. Get the source code:
git clone https://github.com/nkanaev/yarr.git
Then run one of the corresponding commands:
# create an executable for the host os
make build_macos # -> _output/macos/yarr.app
make build_linux # -> _output/linux/yarr
make build_windows # -> _output/windows/yarr.exe
# ... or start a dev server locally
make serve # starts a server at http://localhost:7070
# ... or build a docker image
docker build -t yarr .
For building from source code, see [build.md](build.md)
## credits

View File

@@ -6,6 +6,7 @@ import (
"io"
"io/fs"
"io/ioutil"
"log"
"os"
)
@@ -29,9 +30,18 @@ func Template(path string) *template.Template {
if !found {
tmpl = template.Must(template.New(path).Delims("{%", "%}").Funcs(template.FuncMap{
"inline": func(svg string) template.HTML {
svgfile, _ := FS.Open("graphicarts/" + svg)
content, _ := ioutil.ReadAll(svgfile)
svgfile.Close()
svgfile, err := FS.Open("graphicarts/" + svg)
// should never happen
if err != nil {
log.Fatal(err)
}
defer svgfile.Close()
content, err := ioutil.ReadAll(svgfile)
// should never happen
if err != nil {
log.Fatal(err)
}
return template.HTML(content)
},
}).ParseFS(FS, path))

View File

@@ -1,3 +1,4 @@
//go:build release
// +build release
package assets

View File

@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-chevron-down"><polyline points="6 9 12 15 18 9"></polyline></svg>

Before

Width:  |  Height:  |  Size: 269 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 KiB

View File

@@ -0,0 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-anchor-favicon">
<circle cx="12" cy="5" r="3" stroke-width="4" stroke="#ffffff"></circle>
<line x1="12" y1="22" x2="12" y2="8" stroke-width="4" stroke="#ffffff"></line>
<path d="M5 12H2a10 10 0 0 0 20 0h-3" stroke-width="4" stroke="#ffffff"></path>
<circle cx="12" cy="5" r="3"></circle>
<line x1="12" y1="22" x2="12" y2="8"></line>
<path d="M5 12H2a10 10 0 0 0 20 0h-3"></path>
</svg>

After

Width:  |  Height:  |  Size: 603 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.2 KiB

View File

@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-list"><line x1="8" y1="6" x2="21" y2="6"></line><line x1="8" y1="12" x2="21" y2="12"></line><line x1="8" y1="18" x2="21" y2="18"></line><line x1="3" y1="6" x2="3.01" y2="6"></line><line x1="3" y1="12" x2="3.01" y2="12"></line><line x1="3" y1="18" x2="3.01" y2="18"></line></svg>

Before

Width:  |  Height:  |  Size: 482 B

View File

@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-menu"><line x1="3" y1="12" x2="21" y2="12"></line><line x1="3" y1="6" x2="21" y2="6"></line><line x1="3" y1="18" x2="21" y2="18"></line></svg>

Before

Width:  |  Height:  |  Size: 346 B

View File

@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-more-vertical"><circle cx="12" cy="12" r="1"></circle><circle cx="12" cy="5" r="1"></circle><circle cx="12" cy="19" r="1"></circle></svg>

Before

Width:  |  Height:  |  Size: 341 B

View File

@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-settings"><circle cx="12" cy="12" r="3"></circle><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1 0 2.83 2 2 0 0 1-2.83 0l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-2 2 2 2 0 0 1-2-2v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 0 1-2.83 0 2 2 0 0 1 0-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 0 1-2-2 2 2 0 0 1 2-2h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 0 1 0-2.83 2 2 0 0 1 2.83 0l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 0 1 2-2 2 2 0 0 1 2 2v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 0 1 2.83 0 2 2 0 0 1 0 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9a1.65 1.65 0 0 0 1.51 1H21a2 2 0 0 1 2 2 2 2 0 0 1-2 2h-.09a1.65 1.65 0 0 0-1.51 1z"></path></svg>

Before

Width:  |  Height:  |  Size: 1011 B

View File

@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-trash-2"><polyline points="3 6 5 6 21 6"></polyline><path d="M19 6v14a2 2 0 0 1-2 2H7a2 2 0 0 1-2-2V6m3 0V4a2 2 0 0 1 2-2h4a2 2 0 0 1 2 2v2"></path><line x1="10" y1="11" x2="10" y2="17"></line><line x1="14" y1="11" x2="14" y2="17"></line></svg>

Before

Width:  |  Height:  |  Size: 448 B

View File

@@ -5,7 +5,9 @@
<title>yarr!</title>
<link rel="stylesheet" href="./static/stylesheets/bootstrap.min.css">
<link rel="stylesheet" href="./static/stylesheets/app.css">
<link rel="icon shortcut" href="./static/graphicarts/icon.png">
<link rel="icon" href="./static/graphicarts/favicon.svg" type="image/svg+xml">
<link rel="alternate icon" href="./static/graphicarts/favicon.png" type="image/png">
<link rel="manifest" href="./manifest.json" />
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<script>
window.app = window.app || {}
@@ -131,8 +133,8 @@
:class="{'d-none': filterSelected
&& !(current.folder.id == folder.id || current.feed.folder_id == folder.id)
&& !filteredFolderStats[folder.id]
&& (!itemSelectedDetails || feedsById[itemSelectedDetails.feed_id].folder_id != folder.id)}">
<input type="radio" name="feed" :value="'folder:'+folder.id" v-model="feedSelected">
&& (!itemSelectedDetails || (feedsById[itemSelectedDetails.feed_id] || {}).folder_id != folder.id)}">
<input type="radio" name="feed" :value="'folder:'+folder.id" v-model="feedSelected" v-if="folder.id">
<div class="selectgroup-label d-flex align-items-center w-100" v-if="folder.id">
<span class="icon mr-2"
:class="{expanded: folder.is_expanded}"
@@ -272,7 +274,7 @@
<span class="icon icon-small mr-1" v-if="item.status=='starred'">{% inline "star-full.svg" %}</span>
</transition>
<small class="flex-fill text-truncate mr-1">
{{ feedsById[item.feed_id].title }}
{{ (feedsById[item.feed_id] || {}).title }}
</small>
<small class="flex-shrink-0"><relative-time v-bind:title="formatDate(item.date)" :val="item.date"/></small>
</div>
@@ -333,17 +335,19 @@
class="content px-4 pt-3 pb-5 border-top overflow-auto"
:class="{'font-serif': theme.font == 'serif', 'font-monospace': theme.font == 'monospace'}"
:style="{'font-size': theme.size + 'rem'}">
<h1><b>{{ itemSelectedDetails.title || 'untitled' }}</b></h1>
<div class="text-muted">
<div>{{ feedsById[itemSelectedDetails.feed_id].title }}</div>
<time>{{ formatDate(itemSelectedDetails.date) }}</time>
<div class="content-wrapper">
<h1><b>{{ itemSelectedDetails.title || 'untitled' }}</b></h1>
<div class="text-muted">
<div>{{ (feedsById[itemSelectedDetails.feed_id] || {}).title }}</div>
<time>{{ formatDate(itemSelectedDetails.date) }}</time>
</div>
<hr>
<div v-if="!itemSelectedReadability">
<img :src="itemSelectedDetails.image" v-if="itemSelectedDetails.image" class="mb-3">
<audio class="w-100" controls v-if="itemSelectedDetails.podcast_url" :src="itemSelectedDetails.podcast_url"></audio>
</div>
<div v-html="itemSelectedContent"></div>
</div>
<hr>
<div v-if="!itemSelectedReadability">
<img :src="itemSelectedDetails.image" v-if="itemSelectedDetails.image" class="mb-3">
<audio class="w-100" controls v-if="itemSelectedDetails.podcast_url" :src="itemSelectedDetails.podcast_url"></audio>
</div>
<div v-html="itemSelectedContent"></div>
</div>
</div>
<modal :open="!!settings" @hide="settings = ''">
@@ -354,7 +358,7 @@
<p class="cursor-default"><b>New Feed</b></p>
<form action="" @submit.prevent="createFeed(event)" class="mt-4">
<label for="feed-url">URL</label>
<input id="feed-url" name="url" type="url" class="form-control" required autocomplete="off" :readonly="feedNewChoice.length > 0">
<input id="feed-url" name="url" type="url" class="form-control" required autocomplete="off" :readonly="feedNewChoice.length > 0" placeholder="https://example.com/feed" v-focus>
<label for="feed-folder" class="mt-3 d-block">
Folder
<a href="#" class="float-right text-decoration-none" @click.prevent="createNewFeedFolder()">new folder</a>

View File

@@ -105,7 +105,7 @@
return api('post', './logout')
},
crawl: function(url) {
return api('get', './page?url=' + url).then(json)
return api('get', './page?url=' + encodeURIComponent(url)).then(json)
}
}
})()

View File

@@ -21,6 +21,12 @@ Vue.directive('scroll', {
},
})
Vue.directive('focus', {
inserted: function(el) {
el.focus()
}
})
Vue.component('drag', {
props: ['width'],
template: '<div class="drag"></div>',
@@ -404,6 +410,7 @@ var vm = new Vue({
refreshItems: function(loadMore) {
if (this.feedSelected === null) {
vm.items = []
vm.itemsHasMore = false
return
}
@@ -413,7 +420,7 @@ var vm = new Vue({
}
this.loading.items = true
return api.items.list(query).then(function(data) {
api.items.list(query).then(function(data) {
if (loadMore) {
vm.items = vm.items.concat(data.list)
} else {
@@ -421,14 +428,28 @@ var vm = new Vue({
}
vm.itemsHasMore = data.has_more
vm.loading.items = false
// load more if there's some space left at the bottom of the item list.
vm.$nextTick(function() {
if (vm.itemsHasMore && !vm.loading.items && vm.itemListCloseToBottom()) {
vm.refreshItems(true)
}
})
})
},
itemListCloseToBottom: function() {
// approx. vertical space at the bottom of the list (loading el & paddings) when 1rem = 16px
var bottomSpace = 70
var scale = (parseFloat(getComputedStyle(document.documentElement).fontSize) || 16) / 16
var el = this.$refs.itemlist
var closeToBottom = (el.scrollHeight - el.scrollTop - el.offsetHeight) < bottomSpace * scale
return closeToBottom
},
loadMoreItems: function(event, el) {
if (!this.itemsHasMore) return
if (this.loading.items) return
var closeToBottom = (el.scrollHeight - el.scrollTop - el.offsetHeight) < 50
if (closeToBottom) this.refreshItems(true)
if (this.itemListCloseToBottom()) this.refreshItems(true)
},
markItemsRead: function() {
var query = this.getItemsQuery()
@@ -496,10 +517,7 @@ var vm = new Vue({
deleteFolder: function(folder) {
if (confirm('Are you sure you want to delete ' + folder.title + '?')) {
api.folders.delete(folder.id).then(function() {
if (vm.feedSelected === 'folder:'+folder.id) {
vm.items = []
vm.feedSelected = ''
}
vm.feedSelected = null
vm.refreshStats()
vm.refreshFeeds()
})
@@ -516,12 +534,7 @@ var vm = new Vue({
deleteFeed: function(feed) {
if (confirm('Are you sure you want to delete ' + feed.title + '?')) {
api.feeds.delete(feed.id).then(function() {
// unselect feed to prevent reading properties of null in template
var isSelected = !vm.feedSelected
|| (vm.feedSelected === 'feed:'+feed.id
|| (feed.folder_id && vm.feedSelected === 'folder:'+feed.folder_id));
if (isSelected) vm.feedSelected = null
vm.feedSelected = null
vm.refreshStats()
vm.refreshFeeds()
})

View File

@@ -165,6 +165,24 @@ var keybindings = {
"3": shortcutFunctions.showAll,
}
var codebindings = {
"KeyO": shortcutFunctions.openItemLink,
"KeyI": shortcutFunctions.toggleReadability,
//"r": shortcutFunctions.toggleItemRead,
//"KeyR": shortcutFunctions.markAllRead,
"KeyS": shortcutFunctions.toggleItemStarred,
"Slash": shortcutFunctions.focusSearch,
"KeyJ": shortcutFunctions.nextItem,
"KeyK": shortcutFunctions.previousItem,
"KeyL": shortcutFunctions.nextFeed,
"KeyH": shortcutFunctions.previousFeed,
"KeyF": shortcutFunctions.scrollForward,
"KeyB": shortcutFunctions.scrollBackward,
"Digit1": shortcutFunctions.showUnread,
"Digit2": shortcutFunctions.showStarred,
"Digit3": shortcutFunctions.showAll,
}
function isTextBox(element) {
var tagName = element.tagName.toLowerCase()
// Input elements that aren't text
@@ -179,10 +197,10 @@ function isTextBox(element) {
document.addEventListener('keydown',function(event) {
// Ignore while focused on text or
// when using modifier keys (to not clash with browser behaviour)
if (isTextBox(event.target) || event.metaKey || event.ctrlKey) {
if (isTextBox(event.target) || event.metaKey || event.ctrlKey || event.altKey) {
return
}
var keybindFunction = keybindings[event.key]
var keybindFunction = keybindings[event.key] || codebindings[event.code]
if (keybindFunction) {
event.preventDefault()
keybindFunction()

View File

@@ -5,7 +5,8 @@
<title>yarr!</title>
<link rel="stylesheet" href="./static/stylesheets/bootstrap.min.css">
<link rel="stylesheet" href="./static/stylesheets/app.css">
<link rel="icon shortcut" href="./static/graphicarts/icon.png">
<link rel="icon" href="./static/graphicarts/favicon.svg" type="image/svg+xml">
<link rel="alternate icon" href="./static/graphicarts/favicon.png" type="image/png">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<style>
form {

View File

@@ -2,8 +2,11 @@
display: none !important;
}
body {
html {
font-size: 15px !important;
}
body {
overscroll-behavior: none;
}
@@ -85,6 +88,10 @@ select.form-control:not([multiple]):not([size]) {
outline: none;
}
.table-compact {
color: unset !important;
}
.table-compact tr td:first-child {
padding-left: 0;
}
@@ -160,7 +167,9 @@ select.form-control:not([multiple]):not([size]) {
opacity: 0;
position: absolute;
z-index: -1;
top: 0; left: 0;
top: 0;
left: 0;
height: 100%;
}
.selectgroup + .selectgroup {
@@ -349,6 +358,11 @@ select.form-control:not([multiple]):not([size]) {
line-height: 1.5;
}
.content-wrapper {
max-width: 60rem;
margin: 0 auto;
}
.content img, .content video {
max-width: 100%;
height: auto;
@@ -416,6 +430,11 @@ select.form-control:not([multiple]):not([size]) {
font-size: 1rem;
}
.content p {
margin-top: 1rem;
margin-bottom: 1rem;
}
/* theme: light */
button.theme-light {
@@ -423,11 +442,11 @@ button.theme-light {
}
a,
.btn-link:hover,
.toolbar-item.active {
.btn-link:hover {
color: #0080d4;
}
.toolbar-item.active,
.dropdown-item.active,
.dropdown-item:active,
.selectgroup input:checked + .selectgroup-label {

View File

@@ -6,6 +6,7 @@ package readability
import (
"bytes"
"errors"
"fmt"
"io"
"math"
@@ -59,6 +60,9 @@ func ExtractContent(page io.Reader) (string, error) {
best = body
break
}
if best == nil {
return "", errors.New("failed to extract content")
}
}
//log.Printf("[Readability] TopCandidate: %v", topCandidate)

View File

@@ -364,7 +364,6 @@ func isBlockedTag(tagName string) bool {
}
/*
One or more strings separated by commas, indicating possible image sources for the user agent to use.
Each string is composed of:
@@ -372,7 +371,6 @@ Each string is composed of:
- Optionally, whitespace followed by one of:
- A width descriptor (a positive integer directly followed by w). The width descriptor is divided by the source size given in the sizes attribute to calculate the effective pixel density.
- A pixel density descriptor (a positive floating point number directly followed by x).
*/
func sanitizeSrcsetAttr(baseURL, value string) string {
var sanitizedSources []string

17
src/content/silo/url.go Normal file
View File

@@ -0,0 +1,17 @@
package silo
import (
"net/url"
"strings"
)
func RedirectURL(link string) string {
if strings.HasPrefix(link, "https://www.google.com/url?") {
if u, err := url.Parse(link); err == nil {
if u2 := u.Query().Get("url"); u2 != "" {
return u2
}
}
}
return link
}

View File

@@ -0,0 +1,24 @@
package silo
import "testing"
func TestRedirectURL(t *testing.T) {
link := "https://www.google.com/url?rct=j&sa=t&url=https://www.cryptoglobe.com/latest/2022/08/investment-strategist-lyn-alden-explains-why-she-is-still-bullish-on-bitcoin-long-term/&ct=ga&cd=CAIyGjlkMjI1NjUyODE3ODFjMDQ6Y29tOmVuOlVT&usg=AOvVaw16C2fJtw6m8QVEbto2HCKK"
want := "https://www.cryptoglobe.com/latest/2022/08/investment-strategist-lyn-alden-explains-why-she-is-still-bullish-on-bitcoin-long-term/"
have := RedirectURL(link)
if have != want {
t.Logf("want: %s", want)
t.Logf("have: %s", have)
t.Fail()
}
link = "https://example.com"
if RedirectURL(link) != link {
t.Fail()
}
link = "https://example.com/url?url=test.com"
if RedirectURL(link) != link {
t.Fail()
}
}

View File

@@ -4,6 +4,7 @@ import (
"bufio"
"flag"
"fmt"
"io"
"log"
"os"
"path/filepath"
@@ -28,10 +29,25 @@ func opt(envVar, defaultValue string) string {
return defaultValue
}
func parseAuthfile(authfile io.Reader) (username, password string, err error) {
scanner := bufio.NewScanner(authfile)
for scanner.Scan() {
line := scanner.Text()
parts := strings.SplitN(line, ":", 2)
if len(parts) != 2 {
return "", "", fmt.Errorf("wrong syntax (expected `username:password`)")
}
username = parts[0]
password = parts[1]
break
}
return username, password, nil
}
func main() {
platform.FixConsoleIfNeeded()
var addr, db, authfile, certfile, keyfile, basepath, logfile string
var addr, db, authfile, auth, certfile, keyfile, basepath, logfile string
var ver, open bool
flag.CommandLine.SetOutput(os.Stdout)
@@ -46,7 +62,8 @@ func main() {
flag.StringVar(&addr, "addr", opt("YARR_ADDR", "127.0.0.1:7070"), "address to run server on")
flag.StringVar(&basepath, "base", opt("YARR_BASE", ""), "base path of the service url")
flag.StringVar(&authfile, "auth-file", opt("YARR_AUTHFILE", ""), "`path` to a file containing username:password")
flag.StringVar(&authfile, "auth-file", opt("YARR_AUTHFILE", ""), "`path` to a file containing username:password. Takes precedence over --auth (or YARR_AUTH)")
flag.StringVar(&auth, "auth", opt("YARR_AUTH", ""), "string with username and password in the format `username:password`")
flag.StringVar(&certfile, "cert-file", opt("YARR_CERTFILE", ""), "`path` to cert file for https")
flag.StringVar(&keyfile, "key-file", opt("YARR_KEYFILE", ""), "`path` to key file for https")
flag.StringVar(&db, "db", opt("YARR_DB", ""), "storage file `path`")
@@ -94,16 +111,14 @@ func main() {
log.Fatal("Failed to open auth file: ", err)
}
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := scanner.Text()
parts := strings.Split(line, ":")
if len(parts) != 2 {
log.Fatalf("Invalid auth: %v (expected `username:password`)", line)
}
username = parts[0]
password = parts[1]
break
username, password, err = parseAuthfile(f)
if err != nil {
log.Fatal("Failed to parse auth file: ", err)
}
} else if auth != "" {
username, password, err = parseAuthfile(strings.NewReader(auth))
if err != nil {
log.Fatal("Failed to parse auth literal: ", err)
}
}

47
src/main_test.go Normal file
View File

@@ -0,0 +1,47 @@
package main
import (
"strings"
"testing"
)
func TestPasswordFromAuthfile(t *testing.T) {
for _, tc := range [...]struct {
authfile string
expectedUsername string
expectedPassword string
expectedError bool
}{
{
authfile: "username:password",
expectedUsername: "username",
expectedPassword: "password",
expectedError: false,
},
{
authfile: "username-and-no-password",
expectedError: true,
},
{
authfile: "username:password:with:columns",
expectedUsername: "username",
expectedPassword: "password:with:columns",
expectedError: false,
},
} {
t.Run(tc.authfile, func(t *testing.T) {
username, password, err := parseAuthfile(strings.NewReader(tc.authfile))
if tc.expectedUsername != username {
t.Errorf("expected username %q, got %q", tc.expectedUsername, username)
}
if tc.expectedPassword != password {
t.Errorf("expected password %q, got %q", tc.expectedPassword, password)
}
if tc.expectedError && err == nil {
t.Errorf("expected error, got nil")
} else if !tc.expectedError && err != nil {
t.Errorf("unexpected error: %v", err)
}
})
}
}

View File

@@ -11,18 +11,23 @@ import (
"time"
"github.com/nkanaev/yarr/src/content/htmlutil"
"golang.org/x/net/html/charset"
)
var UnknownFormat = errors.New("unknown feed format")
type processor func(r io.Reader) (*Feed, error)
type feedProbe struct {
feedType string
callback func(r io.Reader) (*Feed, error)
encoding string
}
func sniff(lookup string) (string, processor) {
func sniff(lookup string) (out feedProbe) {
lookup = strings.TrimSpace(lookup)
lookup = strings.TrimLeft(lookup, "\x00\xEF\xBB\xBF\xFE\xFF")
if len(lookup) < 0 {
return "", nil
if len(lookup) == 0 {
return
}
switch lookup[0] {
@@ -33,24 +38,42 @@ func sniff(lookup string) (string, processor) {
if token == nil {
break
}
// check <?xml encoding="ENCODING" ?>
if el, ok := token.(xml.ProcInst); ok && el.Target == "xml" {
out.encoding = strings.ToLower(procInst("encoding", string(el.Inst)))
}
if el, ok := token.(xml.StartElement); ok {
switch el.Name.Local {
case "rss":
return "rss", ParseRSS
out.feedType = "rss"
out.callback = ParseRSS
return
case "RDF":
return "rdf", ParseRDF
out.feedType = "rdf"
out.callback = ParseRDF
return
case "feed":
return "atom", ParseAtom
out.feedType = "atom"
out.callback = ParseAtom
return
}
}
}
case '{':
return "json", ParseJSON
out.feedType = "json"
out.callback = ParseJSON
return
}
return "", nil
return
}
func Parse(r io.Reader) (*Feed, error) {
return ParseWithEncoding(r, "")
}
func ParseWithEncoding(r io.Reader, fallbackEncoding string) (*Feed, error) {
lookup := make([]byte, 2048)
n, err := io.ReadFull(r, lookup)
switch {
@@ -63,18 +86,42 @@ func Parse(r io.Reader) (*Feed, error) {
r = io.MultiReader(bytes.NewReader(lookup), r)
}
_, callback := sniff(string(lookup))
if callback == nil {
out := sniff(string(lookup))
if out.feedType == "" {
return nil, UnknownFormat
}
feed, err := callback(r)
if out.encoding == "" && fallbackEncoding != "" {
r, err = charset.NewReaderLabel(fallbackEncoding, r)
if err != nil {
return nil, err
}
}
if (out.feedType != "json") && (out.encoding == "" || out.encoding == "utf-8") {
// XML decoder will not rely on custom CharsetReader (see `xmlDecoder`)
// to handle invalid xml characters.
// Assume input is already UTF-8 and do the cleanup here.
r = NewSafeXMLReader(r)
}
feed, err := out.callback(r)
if feed != nil {
feed.cleanup()
}
return feed, err
}
func ParseAndFix(r io.Reader, baseURL, fallbackEncoding string) (*Feed, error) {
feed, err := ParseWithEncoding(r, fallbackEncoding)
if err != nil {
return nil, err
}
feed.TranslateURLs(baseURL)
feed.SetMissingDatesTo(time.Now())
return feed, nil
}
func (feed *Feed) cleanup() {
feed.Title = strings.TrimSpace(feed.Title)
feed.SiteURL = strings.TrimSpace(feed.SiteURL)

View File

@@ -7,38 +7,40 @@ import (
)
func TestSniff(t *testing.T) {
testcases := [][2]string{
testcases := []struct {
input string
want feedProbe
}{
{
`<?xml version="1.0"?><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"></rdf:RDF>`,
"rdf",
feedProbe{feedType: "rdf", callback: ParseRDF},
},
{
`<?xml version="1.0" encoding="ISO-8859-1"?><rss version="2.0"><channel></channel></rss>`,
"rss",
feedProbe{feedType: "rss", callback: ParseRSS, encoding: "iso-8859-1"},
},
{
`<?xml version="1.0"?><rss version="2.0"><channel></channel></rss>`,
"rss",
feedProbe{feedType: "rss", callback: ParseRSS},
},
{
`<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"></feed>`,
"atom",
feedProbe{feedType: "atom", callback: ParseAtom, encoding: "utf-8"},
},
{
`{}`,
"json",
feedProbe{feedType: "json", callback: ParseJSON},
},
{
`<!DOCTYPE html><html><head><title></title></head><body></body></html>`,
"",
feedProbe{},
},
}
for _, testcase := range testcases {
have, _ := sniff(testcase[0])
want := testcase[1]
if want != have {
t.Log(testcase[0])
t.Errorf("Invalid format: want=%#v have=%#v", want, have)
want := testcase.want
have := sniff(testcase.input)
if want.encoding != have.encoding || want.feedType != have.feedType {
t.Errorf("Invalid output\n---\n%s\n---\n\nwant=%#v\nhave=%#v", testcase.input, want, have)
}
}
}
@@ -107,3 +109,44 @@ func TestParseFeedWithBOM(t *testing.T) {
t.FailNow()
}
}
func TestParseCleanIllegalCharsInUTF8(t *testing.T) {
data := `
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<item>
<title>` + "\a" + `title</title>
</item>
</channel>
</rss>
`
feed, err := Parse(strings.NewReader(data))
if err != nil {
t.Fatal(err)
}
if len(feed.Items) != 1 || feed.Items[0].Title != "title" {
t.Fatalf("invalid feed, got: %v", feed)
}
}
func TestParseCleanIllegalCharsInNonUTF8(t *testing.T) {
// echo привет | iconv -f utf8 -t cp1251 | hexdump -C
data := `
<?xml version="1.0" encoding="windows-1251"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<item>
<title>` + "\a \xef\xf0\xe8\xe2\xe5\xf2\x0a \a" + `</title>
</item>
</channel>
</rss>
`
feed, err := Parse(strings.NewReader(data))
if err != nil {
t.Fatal(err)
}
if len(feed.Items) != 1 || feed.Items[0].Title != "привет" {
t.Fatalf("invalid feed, got: %v", feed)
}
}

View File

@@ -20,9 +20,9 @@ type rssFeed struct {
}
type rssItem struct {
GUID string `xml:"guid"`
GUID rssGuid `xml:"guid"`
Title string `xml:"title"`
Link string `xml:"link"`
Link string `xml:"rss link"`
Description string `xml:"rss description"`
PubDate string `xml:"pubDate"`
Enclosures []rssEnclosure `xml:"enclosure"`
@@ -36,6 +36,11 @@ type rssItem struct {
media
}
type rssGuid struct {
GUID string `xml:",chardata"`
IsPermaLink string `xml:"isPermaLink,attr"`
}
type rssLink struct {
XMLName xml.Name
Data string `xml:",chardata"`
@@ -81,10 +86,15 @@ func ParseRSS(r io.Reader) (*Feed, error) {
}
}
permalink := ""
if srcitem.GUID.IsPermaLink == "true" {
permalink = srcitem.GUID.GUID
}
dstfeed.Items = append(dstfeed.Items, Item{
GUID: firstNonEmpty(srcitem.GUID, srcitem.Link),
GUID: firstNonEmpty(srcitem.GUID.GUID, srcitem.Link),
Date: dateParse(firstNonEmpty(srcitem.DublinCoreDate, srcitem.PubDate)),
URL: firstNonEmpty(srcitem.OrigLink, srcitem.Link),
URL: firstNonEmpty(srcitem.OrigLink, srcitem.Link, permalink),
Title: srcitem.Title,
Content: firstNonEmpty(srcitem.ContentEncoded, srcitem.Description),
AudioURL: podcastURL,

View File

@@ -203,3 +203,28 @@ func TestRSSTitleHTMLTags(t *testing.T) {
}
}
}
func TestRSSIsPermalink(t *testing.T) {
feed, _ := Parse(strings.NewReader(`
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<item>
<guid isPermaLink="true">http://example.com/posts/1</guid>
</item>
</channel>
</rss>
`))
have := feed.Items
want := []Item{
{
GUID: "http://example.com/posts/1",
URL: "http://example.com/posts/1",
},
}
for i := 0; i < len(want); i++ {
if want[i] != have[i] {
t.Errorf("Failed to handle isPermalink\nwant: %#v\nhave: %#v\n", want[i], have[i])
}
}
}

View File

@@ -1,6 +1,8 @@
package parser
import (
"bufio"
"bytes"
"encoding/xml"
"io"
"regexp"
@@ -30,6 +32,81 @@ func plain2html(text string) string {
func xmlDecoder(r io.Reader) *xml.Decoder {
decoder := xml.NewDecoder(r)
decoder.Strict = false
decoder.CharsetReader = charset.NewReaderLabel
decoder.CharsetReader = func(cs string, input io.Reader) (io.Reader, error) {
r, err := charset.NewReaderLabel(cs, input)
if err == nil {
r = NewSafeXMLReader(r)
}
return r, err
}
return decoder
}
type safexmlreader struct {
reader *bufio.Reader
buffer *bytes.Buffer
}
func NewSafeXMLReader(r io.Reader) io.Reader {
return &safexmlreader{
reader: bufio.NewReader(r),
buffer: bytes.NewBuffer(make([]byte, 0, 4096)),
}
}
func (xr *safexmlreader) Read(p []byte) (int, error) {
for xr.buffer.Len() < cap(p) {
r, _, err := xr.reader.ReadRune()
if err == io.EOF {
if xr.buffer.Len() == 0 {
return 0, io.EOF
}
break
}
if err != nil {
return 0, err
}
if isInCharacterRange(r) {
xr.buffer.WriteRune(r)
}
}
return xr.buffer.Read(p)
}
// NOTE: copied from "encoding/xml" package
// Decide whether the given rune is in the XML Character Range, per
// the Char production of https://www.xml.com/axml/testaxml.htm,
// Section 2.2 Characters.
func isInCharacterRange(r rune) (inrange bool) {
return r == 0x09 ||
r == 0x0A ||
r == 0x0D ||
r >= 0x20 && r <= 0xD7FF ||
r >= 0xE000 && r <= 0xFFFD ||
r >= 0x10000 && r <= 0x10FFFF
}
// NOTE: copied from "encoding/xml" package
// procInst parses the `param="..."` or `param='...'`
// value out of the provided string, returning "" if not found.
func procInst(param, s string) string {
// TODO: this parsing is somewhat lame and not exact.
// It works for all actual cases, though.
param = param + "="
idx := strings.Index(s, param)
if idx == -1 {
return ""
}
v := s[idx+len(param):]
if v == "" {
return ""
}
if v[0] != '\'' && v[0] != '"' {
return ""
}
idx = strings.IndexRune(v[1:], rune(v[0]))
if idx == -1 {
return ""
}
return v[1 : idx+1]
}

88
src/parser/util_test.go Normal file
View File

@@ -0,0 +1,88 @@
package parser
import (
"bytes"
"io"
"reflect"
"testing"
)
func TestSafeXMLReader(t *testing.T) {
var f io.Reader
want := []byte("привет мир")
f = bytes.NewReader(want)
f = NewSafeXMLReader(f)
have, err := io.ReadAll(f)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(want, have) {
t.Fatalf("invalid output\nwant: %v\nhave: %v", want, have)
}
}
func TestSafeXMLReaderRemoveUnwantedRunes(t *testing.T) {
var f io.Reader
input := []byte("\aпривет \x0cмир\ufffe\uffff")
want := []byte("привет мир")
f = bytes.NewReader(input)
f = NewSafeXMLReader(f)
have, err := io.ReadAll(f)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(want, have) {
t.Fatalf("invalid output\nwant: %v\nhave: %v", want, have)
}
}
func TestSafeXMLReaderPartial1(t *testing.T) {
var f io.Reader
input := []byte("\aпривет \x0cмир\ufffe\uffff")
want := []byte("привет мир")
f = bytes.NewReader(input)
f = NewSafeXMLReader(f)
buf := make([]byte, 1)
for i := 0; i < len(want); i++ {
n, err := f.Read(buf)
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Fatalf("expected 1 byte, got %d", n)
}
if buf[0] != want[i] {
t.Fatalf("invalid char at pos %d\nwant: %v\nhave: %v", i, want[i], buf[0])
}
}
if x, err := f.Read(buf); err != io.EOF {
t.Fatalf("expected EOF, %v, %v %v", buf, x, err)
}
}
func TestSafeXMLReaderPartial2(t *testing.T) {
var f io.Reader
input := []byte("привет\a\a\a\a\a")
f = bytes.NewReader(input)
f = NewSafeXMLReader(f)
buf := make([]byte, 12)
n, err := f.Read(buf)
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
if n != 12 {
t.Fatalf("expected 12 bytes")
}
n, err = f.Read(buf)
if n != 0 {
t.Fatalf("expected 0")
}
if err != io.EOF {
t.Fatalf("expected EOF, got %v", err)
}
}

View File

@@ -1,3 +1,4 @@
//go:build !windows
// +build !windows
package platform

View File

@@ -58,17 +58,20 @@ var oldStdin, oldStdout, oldStderr *os.File
//
// Net result is as follows.
// Before:
// SHELL NON-REDIRECTED REDIRECTED
// explorer.exe no console n/a
// cmd.exe broken works
// powershell broken broken
// WSL bash broken works
//
// SHELL NON-REDIRECTED REDIRECTED
// explorer.exe no console n/a
// cmd.exe broken works
// powershell broken broken
// WSL bash broken works
//
// After
// SHELL NON-REDIRECTED REDIRECTED
// explorer.exe no console n/a
// cmd.exe works works
// powershell works broken
// WSL bash works works
//
// SHELL NON-REDIRECTED REDIRECTED
// explorer.exe no console n/a
// cmd.exe works works
// powershell works broken
// WSL bash works works
//
// We don't seem to make anything worse, at least.
func FixConsoleIfNeeded() error {

View File

@@ -1,3 +1,4 @@
//go:build macos || windows
// +build macos windows
package platform

View File

@@ -1,3 +1,4 @@
//go:build !windows && !macos
// +build !windows,!macos
package platform

View File

@@ -1,3 +1,4 @@
//go:build macos
// +build macos
package platform

View File

@@ -1,3 +1,4 @@
//go:build windows
// +build windows
package platform

View File

@@ -1,3 +1,4 @@
//go:build !windows && !darwin
// +build !windows,!darwin
package platform

View File

@@ -1,3 +1,4 @@
//go:build darwin
// +build darwin
package platform

View File

@@ -1,3 +1,4 @@
//go:build windows
// +build windows
package platform

View File

@@ -3,6 +3,8 @@ package opml
import (
"encoding/xml"
"io"
"golang.org/x/net/html/charset"
)
type opml struct {
@@ -45,6 +47,7 @@ func Parse(r io.Reader) (Folder, error) {
decoder := xml.NewDecoder(r)
decoder.Entity = xml.HTMLEntity
decoder.Strict = false
decoder.CharsetReader = charset.NewReaderLabel
err := decoder.Decode(&val)
if err != nil {

View File

@@ -1,6 +1,7 @@
package opml
import (
"os"
"reflect"
"strings"
"testing"
@@ -87,3 +88,41 @@ func TestParseFallback(t *testing.T) {
t.Fatal("invalid opml")
}
}
func TestParseWithEncoding(t *testing.T) {
file, err := os.Open("sample_win1251.xml")
if err != nil {
t.Fatal(err)
}
have, err := Parse(file)
if err != nil {
t.Fatal(err)
}
want := Folder{
Title: "",
Feeds: []Feed{
{
Title: "пример1",
FeedUrl: "https://baz.com/feed.xml",
SiteUrl: "https://baz.com/",
},
},
Folders: []Folder{
{
Title: "папка",
Feeds: []Feed{
{
Title: "пример2",
FeedUrl: "https://foo.com/feed.xml",
SiteUrl: "https://foo.com/",
},
},
},
},
}
if !reflect.DeepEqual(want, have) {
t.Logf("want: %#v", want)
t.Logf("have: %#v", have)
t.Fatal("invalid opml")
}
}

View File

@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="windows-1251"?>
<opml version="1.1">
<head><title><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD></title></head>
<body>
<outline text="<22><><EFBFBD><EFBFBD><EFBFBD>">
<outline type="rss" text="<22><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>2" description="<22><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>2" xmlUrl="https://foo.com/feed.xml" htmlUrl="https://foo.com/"/>
</outline>
<outline type="rss" text="<22><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>1" description="<22><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>1" xmlUrl="https://baz.com/feed.xml" htmlUrl="https://baz.com/"/>
</body>
</opml>

View File

@@ -32,10 +32,13 @@ func (r *Router) Use(h Handler) {
}
func (r *Router) For(path string, handler Handler) {
chain := make([]Handler, 0)
chain = append(chain, r.middle...)
chain = append(chain, handler)
x := Route{}
x.regex = routeRegexp(path)
x.chain = append(r.middle, handler)
x.chain = chain
r.routes = append(r.routes, x)
}

View File

@@ -39,6 +39,7 @@ func (s *Server) handler() http.Handler {
}
r.For("/", s.handleIndex)
r.For("/manifest.json", s.handleManifest)
r.For("/static/*path", s.handleStatic)
r.For("/api/status", s.handleStatus)
r.For("/api/folders", s.handleFolderList)
@@ -76,6 +77,24 @@ func (s *Server) handleStatic(c *router.Context) {
http.StripPrefix(s.BasePath+"/static/", http.FileServer(http.FS(assets.FS))).ServeHTTP(c.Out, c.Req)
}
func (s *Server) handleManifest(c *router.Context) {
c.JSON(http.StatusOK, map[string]interface{}{
"$schema": "https://json.schemastore.org/web-manifest-combined.json",
"name": "yarr!",
"short_name": "yarr",
"description": "yet another rss reader",
"display": "standalone",
"start_url": s.BasePath,
"icons": []map[string]interface{}{
{
"src": s.BasePath + "/static/graphicarts/favicon.png",
"sizes": "64x64",
"type": "image/png",
},
},
})
}
func (s *Server) handleStatus(c *router.Context) {
c.JSON(http.StatusOK, map[string]interface{}{
"running": s.worker.FeedsPending(),
@@ -159,7 +178,9 @@ func (s *Server) handleFeedIcon(c *router.Context) {
}
cachekey := "icon:" + strconv.FormatInt(id, 10)
s.cache_mutex.Lock()
cachedat := s.cache[cachekey]
s.cache_mutex.Unlock()
if cachedat == nil {
feed := s.db.GetFeed(id)
if feed == nil || feed.Icon == nil {
@@ -177,7 +198,9 @@ func (s *Server) handleFeedIcon(c *router.Context) {
bytes: *(*feed).Icon,
etag: etag,
}
s.cache_mutex.Lock()
s.cache[cachekey] = cachedat
s.cache_mutex.Unlock()
}
icon := cachedat.(feedicon)
@@ -219,7 +242,12 @@ func (s *Server) handleFeedList(c *router.Context) {
result.FeedLink,
form.FolderID,
)
s.db.CreateItems(worker.ConvertItems(result.Feed.Items, *feed))
items := worker.ConvertItems(result.Feed.Items, *feed)
if len(items) > 0 {
s.db.CreateItems(items)
s.db.SetFeedSize(feed.Id, len(items))
s.db.SyncSearch()
}
s.worker.FindFeedFavicon(*feed)
c.JSON(http.StatusOK, map[string]interface{}{
@@ -334,7 +362,7 @@ func (s *Server) handleItemList(c *router.Context) {
items = items[:perPage]
}
c.JSON(http.StatusOK, map[string]interface{}{
"list": items,
"list": items,
"has_more": hasMore,
})
} else if c.Req.Method == "PUT" {
@@ -450,6 +478,9 @@ func (s *Server) handleOPMLExport(c *router.Context) {
func (s *Server) handlePageCrawl(c *router.Context) {
url := c.Req.URL.Query().Get("url")
if newUrl := silo.RedirectURL(url); newUrl != "" {
url = newUrl
}
if content := silo.VideoIFrame(url); content != "" {
c.JSON(http.StatusOK, map[string]string{
"content": sanitizer.Sanitize(url, content),
@@ -457,17 +488,17 @@ func (s *Server) handlePageCrawl(c *router.Context) {
return
}
res, err := http.Get(url)
body, err := worker.GetBody(url)
if err != nil {
log.Print(err)
c.Out.WriteHeader(http.StatusBadRequest)
return
}
defer res.Body.Close()
content, err := readability.ExtractContent(res.Body)
content, err := readability.ExtractContent(strings.NewReader(body))
if err != nil {
log.Print(err)
c.Out.WriteHeader(http.StatusNoContent)
c.JSON(http.StatusOK, map[string]string{
"content": "error: " + err.Error(),
})
return
}
content = sanitizer.Sanitize(url, content)

View File

@@ -3,16 +3,18 @@ package server
import (
"log"
"net/http"
"sync"
"github.com/nkanaev/yarr/src/storage"
"github.com/nkanaev/yarr/src/worker"
)
type Server struct {
Addr string
db *storage.Storage
worker *worker.Worker
cache map[string]interface{}
Addr string
db *storage.Storage
worker *worker.Worker
cache map[string]interface{}
cache_mutex *sync.Mutex
BasePath string
@@ -26,10 +28,11 @@ type Server struct {
func NewServer(db *storage.Storage, addr string) *Server {
return &Server{
db: db,
Addr: addr,
worker: worker.NewWorker(db),
cache: make(map[string]interface{}),
db: db,
Addr: addr,
worker: worker.NewWorker(db),
cache: make(map[string]interface{}),
cache_mutex: &sync.Mutex{},
}
}

View File

@@ -20,20 +20,21 @@ func (s *Storage) CreateFeed(title, description, link, feedLink string, folderId
if title == "" {
title = feedLink
}
result, err := s.db.Exec(`
row := s.db.QueryRow(`
insert into feeds (title, description, link, feed_link, folder_id)
values (?, ?, ?, ?, ?)
on conflict (feed_link) do update set folder_id=?`,
on conflict (feed_link) do update set folder_id = ?
returning id`,
title, description, link, feedLink, folderId,
folderId,
)
if err != nil {
return nil
}
id, idErr := result.LastInsertId()
if idErr != nil {
return nil
}
var id int64
err := row.Scan(&id)
if err != nil {
log.Print(err)
return nil
}
return &Feed{
Id: id,
Title: title,
@@ -194,3 +195,15 @@ func (s *Storage) GetFeedErrors() map[int64]string {
}
return errors
}
func (s *Storage) SetFeedSize(feedId int64, size int) {
_, err := s.db.Exec(`
insert into feed_sizes (feed_id, size)
values (?, ?)
on conflict (feed_id) do update set size = excluded.size`,
feedId, size,
)
if err != nil {
log.Print(err)
}
}

View File

@@ -17,6 +17,23 @@ func TestCreateFeed(t *testing.T) {
}
}
func TestCreateFeedSameLink(t *testing.T) {
db := testDB()
feed1 := db.CreateFeed("title", "", "", "http://example1.com/feed.xml", nil)
if feed1 == nil || feed1.Id == 0 {
t.Fatal("expected feed")
}
for i := 0; i < 10; i++ {
db.CreateFeed("title", "", "", "http://example2.com/feed.xml", nil)
}
feed2 := db.CreateFeed("title", "", "http://example.com", "http://example1.com/feed.xml", nil)
if feed1.Id != feed2.Id {
t.Fatalf("expected the same feed.\nwant: %#v\nhave: %#v", feed1, feed2)
}
}
func TestReadFeed(t *testing.T) {
db := testDB()
if db.GetFeed(100500) != nil {

View File

@@ -1,7 +1,6 @@
package storage
import (
"fmt"
"log"
)
@@ -13,35 +12,21 @@ type Folder struct {
func (s *Storage) CreateFolder(title string) *Folder {
expanded := true
result, err := s.db.Exec(`
row := s.db.QueryRow(`
insert into folders (title, is_expanded) values (?, ?)
on conflict (title) do nothing`,
on conflict (title) do update set title = ?
returning id`,
title, expanded,
// provide title again so that we can extract row id
title,
)
if err != nil {
fmt.Println(err)
return nil
}
var id int64
err := row.Scan(&id)
var id int64
numrows, err := result.RowsAffected()
if err != nil {
log.Print(err)
return nil
}
if numrows == 1 {
id, err = result.LastInsertId()
if err != nil {
log.Print(err)
return nil
}
} else {
err = s.db.QueryRow(`select id, is_expanded from folders where title=?`, title).Scan(&id, &expanded)
if err != nil {
log.Print(err)
return nil
}
}
return &Folder{Id: id, Title: title, IsExpanded: expanded}
}

View File

@@ -61,7 +61,7 @@ type ItemFilter struct {
FeedID *int64
Status *ItemStatus
Search *string
After *int64
After *int64
}
type MarkFilter struct {
@@ -76,7 +76,7 @@ func (s *Storage) CreateItems(items []Item) bool {
return false
}
now := time.Now()
now := time.Now().UTC()
for _, item := range items {
_, err = tx.Exec(`
@@ -85,7 +85,7 @@ func (s *Storage) CreateItems(items []Item) bool {
content, image, podcast_url,
date_arrived, status
)
values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
values (?, ?, ?, ?, strftime('%Y-%m-%d %H:%M:%f', ?), ?, ?, ?, ?, ?)
on conflict (feed_id, guid) do nothing`,
item.GUID, item.FeedId, item.Title, item.Link, item.Date,
item.Content, item.ImageURL, item.AudioURL,
@@ -292,45 +292,70 @@ func (s *Storage) SyncSearch() {
}
}
var (
itemsKeepSize = 50
itemsKeepDays = 90
)
// Delete old articles from the database to cleanup space.
//
// The rules:
// - Never delete starred entries.
// - Keep at least the same amount of articles the feed provides (default: 50).
// This prevents from deleting items for rarely updated and/or ever-growing
// feeds which might eventually reappear as unread.
// - Keep entries for a certain period (default: 90 days).
func (s *Storage) DeleteOldItems() {
rows, err := s.db.Query(fmt.Sprintf(`
select feed_id, count(*) as num_items
from items
where status != %d
group by feed_id
having num_items > 50
`, STARRED))
rows, err := s.db.Query(`
select
i.feed_id,
max(coalesce(s.size, 0), ?) as max_items,
count(*) as num_items
from items i
left outer join feed_sizes s on s.feed_id = i.feed_id
where status != ?
group by i.feed_id
`, itemsKeepSize, STARRED)
if err != nil {
log.Print(err)
return
}
feedIds := make([]int64, 0)
feedLimits := make(map[int64]int64, 0)
for rows.Next() {
var id int64
rows.Scan(&id, nil)
feedIds = append(feedIds, id)
var feedId, limit int64
rows.Scan(&feedId, &limit, nil)
feedLimits[feedId] = limit
}
for _, feedId := range feedIds {
for feedId, limit := range feedLimits {
result, err := s.db.Exec(`
delete from items where feed_id = ? and status != ? and date_arrived < ?`,
delete from items
where id in (
select i.id
from items i
where i.feed_id = ? and status != ?
order by date desc
limit -1 offset ?
) and date_arrived < ?
`,
feedId,
STARRED,
time.Now().Add(-time.Hour*24*90), // 90 days
limit,
time.Now().UTC().Add(-time.Hour*time.Duration(24*itemsKeepDays)),
)
if err != nil {
log.Print(err)
return
}
num, err := result.RowsAffected()
numDeleted, err := result.RowsAffected()
if err != nil {
log.Print(err)
return
}
if num > 0 {
log.Printf("Deleted %d old items (%d)", num, feedId)
if numDeleted > 0 {
log.Printf("Deleted %d old items (feed: %d)", numDeleted, feedId)
}
}
}

View File

@@ -3,6 +3,7 @@ package storage
import (
"log"
"reflect"
"strconv"
"testing"
"time"
)
@@ -45,14 +46,14 @@ func testItemsSetup(db *Storage) testItemScope {
db.CreateItems([]Item{
// feed11
{GUID: "item111", FeedId: feed11.Id, Title: "title111", Date: now.Add(time.Hour * 24 * 1)},
{GUID: "item112", FeedId: feed11.Id, Title: "title112", Date: now.Add(time.Hour * 24 * 2)}, // read
{GUID: "item113", FeedId: feed11.Id, Title: "title113", Date: now.Add(time.Hour * 24 * 3)}, // starred
{GUID: "item112", FeedId: feed11.Id, Title: "title112", Date: now.Add(time.Hour * 24 * 2)}, // read
{GUID: "item113", FeedId: feed11.Id, Title: "title113", Date: now.Add(time.Hour * 24 * 3)}, // starred
// feed12
{GUID: "item121", FeedId: feed12.Id, Title: "title121", Date: now.Add(time.Hour * 24 * 4)},
{GUID: "item122", FeedId: feed12.Id, Title: "title122", Date: now.Add(time.Hour * 24 * 5)}, // read
{GUID: "item122", FeedId: feed12.Id, Title: "title122", Date: now.Add(time.Hour * 24 * 5)}, // read
// feed21
{GUID: "item211", FeedId: feed21.Id, Title: "title211", Date: now.Add(time.Hour * 24 * 6)}, // read
{GUID: "item212", FeedId: feed21.Id, Title: "title212", Date: now.Add(time.Hour * 24 * 7)}, // starred
{GUID: "item211", FeedId: feed21.Id, Title: "title211", Date: now.Add(time.Hour * 24 * 6)}, // read
{GUID: "item212", FeedId: feed21.Id, Title: "title212", Date: now.Add(time.Hour * 24 * 7)}, // starred
// feed01
{GUID: "item011", FeedId: feed01.Id, Title: "title011", Date: now.Add(time.Hour * 24 * 8)},
{GUID: "item012", FeedId: feed01.Id, Title: "title012", Date: now.Add(time.Hour * 24 * 9)}, // read
@@ -271,3 +272,59 @@ func TestMarkItemsRead(t *testing.T) {
t.Fail()
}
}
func TestDeleteOldItems(t *testing.T) {
extraItems := 10
now := time.Now().UTC()
db := testDB()
feed := db.CreateFeed("feed", "", "", "http://test.com/feed11.xml", nil)
items := make([]Item, 0)
for i := 0; i < itemsKeepSize+extraItems; i++ {
istr := strconv.Itoa(i)
items = append(items, Item{
GUID: istr,
FeedId: feed.Id,
Title: istr,
Date: now.Add(time.Hour * time.Duration(i)),
})
}
db.CreateItems(items)
db.SetFeedSize(feed.Id, itemsKeepSize)
var feedSize int
err := db.db.QueryRow(
`select size from feed_sizes where feed_id = ?`, feed.Id,
).Scan(&feedSize)
if err != nil {
t.Fatal(err)
}
if feedSize != itemsKeepSize {
t.Fatalf(
"expected feed size to get updated\nwant: %d\nhave: %d",
itemsKeepSize+extraItems,
feedSize,
)
}
// expire only the first 3 articles
_, err = db.db.Exec(
`update items set date_arrived = ?
where id in (select id from items limit 3)`,
now.Add(-time.Hour*time.Duration(itemsKeepDays*24)),
)
if err != nil {
t.Fatal(err)
}
db.DeleteOldItems()
feedItems := db.ListItems(ItemFilter{FeedID: &feed.Id}, 1000, false)
if len(feedItems) != len(items)-3 {
t.Fatalf(
"invalid number of old items kept\nwant: %d\nhave: %d",
len(items)-3,
len(feedItems),
)
}
}

View File

@@ -4,6 +4,7 @@ import (
"database/sql"
"fmt"
"log"
"time"
)
var migrations = []func(*sql.Tx) error{
@@ -13,6 +14,8 @@ var migrations = []func(*sql.Tx) error{
m04_item_podcasturl,
m05_move_description_to_content,
m06_fill_missing_dates,
m07_add_feed_size,
m08_normalize_datetime,
}
var maxVersion = int64(len(migrations))
@@ -259,3 +262,35 @@ func m06_fill_missing_dates(tx *sql.Tx) error {
_, err := tx.Exec(sql)
return err
}
func m07_add_feed_size(tx *sql.Tx) error {
sql := `
create table if not exists feed_sizes (
feed_id references feeds(id) on delete cascade unique,
size integer not null default 0
);
`
_, err := tx.Exec(sql)
return err
}
func m08_normalize_datetime(tx *sql.Tx) error {
rows, err := tx.Query(`select id, date_arrived from items;`)
if err != nil {
return err
}
for rows.Next() {
var id int64
var dateArrived time.Time
err = rows.Scan(&id, &dateArrived)
if err != nil {
return err
}
_, err = tx.Exec(`update items set date_arrived = ? where id = ?;`, dateArrived.UTC(), id)
if err != nil {
return err
}
}
_, err = tx.Exec(`update items set date = strftime('%Y-%m-%d %H:%M:%f', date);`)
return err
}

View File

@@ -11,6 +11,7 @@ func testDB() *Storage {
log.SetOutput(io.Discard)
db, _ := New(":memory:")
log.SetOutput(os.Stderr)
log.SetFlags(log.Ldate | log.Ltime | log.Lshortfile)
return db
}

View File

@@ -9,3 +9,4 @@ hash:
changes:
-removed `getlantern/golog` dependency
-prevent from compiling in linux

View File

@@ -1,3 +1,5 @@
// +build darwin windows
/*
Package systray is a cross-platform Go library to place an icon and menu in the notification area.
*/

View File

@@ -1,3 +1,5 @@
// +build never
package systray
/*

View File

@@ -1,9 +1,8 @@
// +build !windows
// +build darwin
package systray
/*
#cgo linux pkg-config: gtk+-3.0 appindicator3-0.1
#cgo darwin CFLAGS: -DDARWIN -x objective-c -fobjc-arc
#cgo darwin LDFLAGS: -framework Cocoa

View File

@@ -6,10 +6,10 @@ import (
"fmt"
"io"
"io/ioutil"
"mime"
"net/http"
"net/url"
"strings"
"time"
"github.com/nkanaev/yarr/src/content/scraper"
"github.com/nkanaev/yarr/src/parser"
@@ -39,29 +39,32 @@ func DiscoverFeed(candidateUrl string) (*DiscoverResult, error) {
if res.StatusCode != 200 {
return nil, fmt.Errorf("status code %d", res.StatusCode)
}
cs := getCharset(res)
body, err := httpBody(res)
if err != nil {
return nil, err
}
content, err := ioutil.ReadAll(body)
body, err := io.ReadAll(res.Body)
if err != nil {
return nil, err
}
// Try to feed into parser
feed, err := parser.Parse(bytes.NewReader(content))
feed, err := parser.ParseAndFix(bytes.NewReader(body), candidateUrl, cs)
if err == nil {
feed.TranslateURLs(candidateUrl)
feed.SetMissingDatesTo(time.Now())
result.Feed = feed
result.FeedLink = candidateUrl
return result, nil
}
// Possibly an html link. Search for feed links
content := string(body)
if cs != "" {
if r, err := charset.NewReaderLabel(cs, bytes.NewReader(body)); err == nil {
if body, err := io.ReadAll(r); err == nil {
content = string(body)
}
}
}
sources := make([]FeedSource, 0)
for url, title := range scraper.FindFeeds(string(content), candidateUrl) {
for url, title := range scraper.FindFeeds(content, candidateUrl) {
sources = append(sources, FeedSource{Title: title, Url: url})
}
switch {
@@ -187,12 +190,7 @@ func listItems(f storage.Feed, db *storage.Storage) ([]storage.Item, error) {
return nil, nil
}
body, err := httpBody(res)
if err != nil {
return nil, err
}
feed, err := parser.Parse(body)
feed, err := parser.ParseAndFix(res.Body, f.FeedLink, getCharset(res))
if err != nil {
return nil, err
}
@@ -202,15 +200,42 @@ func listItems(f storage.Feed, db *storage.Storage) ([]storage.Item, error) {
if lmod != "" || etag != "" {
db.SetHTTPState(f.Id, lmod, etag)
}
feed.TranslateURLs(f.FeedLink)
feed.SetMissingDatesTo(time.Now())
return ConvertItems(feed.Items, f), nil
}
func httpBody(res *http.Response) (io.Reader, error) {
func getCharset(res *http.Response) string {
contentType := res.Header.Get("Content-Type")
if _, params, err := mime.ParseMediaType(contentType); err == nil {
if cs, ok := params["charset"]; ok {
if e, _ := charset.Lookup(cs); e != nil {
return cs
}
}
}
return ""
}
func GetBody(url string) (string, error) {
res, err := client.get(url)
if err != nil {
return "", err
}
defer res.Body.Close()
var r io.Reader
ctype := res.Header.Get("Content-Type")
if strings.Contains(ctype, "charset") {
return charset.NewReader(res.Body, ctype)
r, err = charset.NewReader(res.Body, ctype)
if err != nil {
return "", err
}
} else {
r = res.Body
}
return res.Body, nil
body, err := io.ReadAll(r)
if err != nil {
return "", err
}
return string(body), nil
}

View File

@@ -121,7 +121,11 @@ func (w *Worker) refresher(feeds []storage.Feed) {
srcqueue <- feed
}
for i := 0; i < len(feeds); i++ {
w.db.CreateItems(<-dstqueue)
items := <-dstqueue
if len(items) > 0 {
w.db.CreateItems(items)
w.db.SetFeedSize(items[0].FeedId, len(items))
}
atomic.AddInt32(w.pending, -1)
w.db.SyncSearch()
}

4
vendor/github.com/mattn/go-sqlite3/.codecov.yml generated vendored Normal file
View File

@@ -0,0 +1,4 @@
coverage:
status:
project: off
patch: off

14
vendor/github.com/mattn/go-sqlite3/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,14 @@
*.db
*.exe
*.dll
*.o
# VSCode
.vscode
# Exclude from upgrade
upgrade/*.c
upgrade/*.h
# Exclude upgrade binary
upgrade/upgrade

21
vendor/github.com/mattn/go-sqlite3/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2014 Yasuhiro Matsumoto
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

592
vendor/github.com/mattn/go-sqlite3/README.md generated vendored Normal file
View File

@@ -0,0 +1,592 @@
go-sqlite3
==========
[![GoDoc Reference](https://godoc.org/github.com/mattn/go-sqlite3?status.svg)](http://godoc.org/github.com/mattn/go-sqlite3)
[![GitHub Actions](https://github.com/mattn/go-sqlite3/workflows/Go/badge.svg)](https://github.com/mattn/go-sqlite3/actions?query=workflow%3AGo)
[![Financial Contributors on Open Collective](https://opencollective.com/mattn-go-sqlite3/all/badge.svg?label=financial+contributors)](https://opencollective.com/mattn-go-sqlite3)
[![codecov](https://codecov.io/gh/mattn/go-sqlite3/branch/master/graph/badge.svg)](https://codecov.io/gh/mattn/go-sqlite3)
[![Go Report Card](https://goreportcard.com/badge/github.com/mattn/go-sqlite3)](https://goreportcard.com/report/github.com/mattn/go-sqlite3)
Latest stable version is v1.14 or later not v2.
~~**NOTE:** The increase to v2 was an accident. There were no major changes or features.~~
# Description
sqlite3 driver conforming to the built-in database/sql interface
Supported Golang version: See [.github/workflows/go.yaml](./.github/workflows/go.yaml)
[This package follows the official Golang Release Policy.](https://golang.org/doc/devel/release.html#policy)
### Overview
- [go-sqlite3](#go-sqlite3)
- [Description](#description)
- [Overview](#overview)
- [Installation](#installation)
- [API Reference](#api-reference)
- [Connection String](#connection-string)
- [DSN Examples](#dsn-examples)
- [Features](#features)
- [Usage](#usage)
- [Feature / Extension List](#feature--extension-list)
- [Compilation](#compilation)
- [Android](#android)
- [ARM](#arm)
- [Cross Compile](#cross-compile)
- [Google Cloud Platform](#google-cloud-platform)
- [Linux](#linux)
- [Alpine](#alpine)
- [Fedora](#fedora)
- [Ubuntu](#ubuntu)
- [Mac OSX](#mac-osx)
- [Windows](#windows)
- [Errors](#errors)
- [User Authentication](#user-authentication)
- [Compile](#compile)
- [Usage](#usage-1)
- [Create protected database](#create-protected-database)
- [Password Encoding](#password-encoding)
- [Available Encoders](#available-encoders)
- [Restrictions](#restrictions)
- [Support](#support)
- [User Management](#user-management)
- [SQL](#sql)
- [Examples](#examples)
- [*SQLiteConn](#sqliteconn)
- [Attached database](#attached-database)
- [Extensions](#extensions)
- [Spatialite](#spatialite)
- [FAQ](#faq)
- [License](#license)
- [Author](#author)
# Installation
This package can be installed with the go get command:
go get github.com/mattn/go-sqlite3
_go-sqlite3_ is *cgo* package.
If you want to build your app using go-sqlite3, you need gcc.
However, after you have built and installed _go-sqlite3_ with `go install github.com/mattn/go-sqlite3` (which requires gcc), you can build your app without relying on gcc in future.
***Important: because this is a `CGO` enabled package you are required to set the environment variable `CGO_ENABLED=1` and have a `gcc` compile present within your path.***
# API Reference
API documentation can be found here: http://godoc.org/github.com/mattn/go-sqlite3
Examples can be found under the [examples](./_example) directory
# Connection String
When creating a new SQLite database or connection to an existing one, with the file name additional options can be given.
This is also known as a DSN string. (Data Source Name).
Options are append after the filename of the SQLite database.
The database filename and options are seperated by an `?` (Question Mark).
Options should be URL-encoded (see [url.QueryEscape](https://golang.org/pkg/net/url/#QueryEscape)).
This also applies when using an in-memory database instead of a file.
Options can be given using the following format: `KEYWORD=VALUE` and multiple options can be combined with the `&` ampersand.
This library supports dsn options of SQLite itself and provides additional options.
Boolean values can be one of:
* `0` `no` `false` `off`
* `1` `yes` `true` `on`
| Name | Key | Value(s) | Description |
|------|-----|----------|-------------|
| UA - Create | `_auth` | - | Create User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Username | `_auth_user` | `string` | Username for User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Password | `_auth_pass` | `string` | Password for User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Crypt | `_auth_crypt` | <ul><li>SHA1</li><li>SSHA1</li><li>SHA256</li><li>SSHA256</li><li>SHA384</li><li>SSHA384</li><li>SHA512</li><li>SSHA512</li></ul> | Password encoder to use for User Authentication, for more information see [User Authentication](#user-authentication) |
| UA - Salt | `_auth_salt` | `string` | Salt to use if the configure password encoder requires a salt, for User Authentication, for more information see [User Authentication](#user-authentication) |
| Auto Vacuum | `_auto_vacuum` \| `_vacuum` | <ul><li>`0` \| `none`</li><li>`1` \| `full`</li><li>`2` \| `incremental`</li></ul> | For more information see [PRAGMA auto_vacuum](https://www.sqlite.org/pragma.html#pragma_auto_vacuum) |
| Busy Timeout | `_busy_timeout` \| `_timeout` | `int` | Specify value for sqlite3_busy_timeout. For more information see [PRAGMA busy_timeout](https://www.sqlite.org/pragma.html#pragma_busy_timeout) |
| Case Sensitive LIKE | `_case_sensitive_like` \| `_cslike` | `boolean` | For more information see [PRAGMA case_sensitive_like](https://www.sqlite.org/pragma.html#pragma_case_sensitive_like) |
| Defer Foreign Keys | `_defer_foreign_keys` \| `_defer_fk` | `boolean` | For more information see [PRAGMA defer_foreign_keys](https://www.sqlite.org/pragma.html#pragma_defer_foreign_keys) |
| Foreign Keys | `_foreign_keys` \| `_fk` | `boolean` | For more information see [PRAGMA foreign_keys](https://www.sqlite.org/pragma.html#pragma_foreign_keys) |
| Ignore CHECK Constraints | `_ignore_check_constraints` | `boolean` | For more information see [PRAGMA ignore_check_constraints](https://www.sqlite.org/pragma.html#pragma_ignore_check_constraints) |
| Immutable | `immutable` | `boolean` | For more information see [Immutable](https://www.sqlite.org/c3ref/open.html) |
| Journal Mode | `_journal_mode` \| `_journal` | <ul><li>DELETE</li><li>TRUNCATE</li><li>PERSIST</li><li>MEMORY</li><li>WAL</li><li>OFF</li></ul> | For more information see [PRAGMA journal_mode](https://www.sqlite.org/pragma.html#pragma_journal_mode) |
| Locking Mode | `_locking_mode` \| `_locking` | <ul><li>NORMAL</li><li>EXCLUSIVE</li></ul> | For more information see [PRAGMA locking_mode](https://www.sqlite.org/pragma.html#pragma_locking_mode) |
| Mode | `mode` | <ul><li>ro</li><li>rw</li><li>rwc</li><li>memory</li></ul> | Access Mode of the database. For more information see [SQLite Open](https://www.sqlite.org/c3ref/open.html) |
| Mutex Locking | `_mutex` | <ul><li>no</li><li>full</li></ul> | Specify mutex mode. |
| Query Only | `_query_only` | `boolean` | For more information see [PRAGMA query_only](https://www.sqlite.org/pragma.html#pragma_query_only) |
| Recursive Triggers | `_recursive_triggers` \| `_rt` | `boolean` | For more information see [PRAGMA recursive_triggers](https://www.sqlite.org/pragma.html#pragma_recursive_triggers) |
| Secure Delete | `_secure_delete` | `boolean` \| `FAST` | For more information see [PRAGMA secure_delete](https://www.sqlite.org/pragma.html#pragma_secure_delete) |
| Shared-Cache Mode | `cache` | <ul><li>shared</li><li>private</li></ul> | Set cache mode for more information see [sqlite.org](https://www.sqlite.org/sharedcache.html) |
| Synchronous | `_synchronous` \| `_sync` | <ul><li>0 \| OFF</li><li>1 \| NORMAL</li><li>2 \| FULL</li><li>3 \| EXTRA</li></ul> | For more information see [PRAGMA synchronous](https://www.sqlite.org/pragma.html#pragma_synchronous) |
| Time Zone Location | `_loc` | auto | Specify location of time format. |
| Transaction Lock | `_txlock` | <ul><li>immediate</li><li>deferred</li><li>exclusive</li></ul> | Specify locking behavior for transactions. |
| Writable Schema | `_writable_schema` | `Boolean` | When this pragma is on, the SQLITE_MASTER tables in which database can be changed using ordinary UPDATE, INSERT, and DELETE statements. Warning: misuse of this pragma can easily result in a corrupt database file. |
| Cache Size | `_cache_size` | `int` | Maximum cache size; default is 2000K (2M). See [PRAGMA cache_size](https://sqlite.org/pragma.html#pragma_cache_size) |
## DSN Examples
```
file:test.db?cache=shared&mode=memory
```
# Features
This package allows additional configuration of features available within SQLite3 to be enabled or disabled by golang build constraints also known as build `tags`.
[Click here for more information about build tags / constraints.](https://golang.org/pkg/go/build/#hdr-Build_Constraints)
### Usage
If you wish to build this library with additional extensions / features.
Use the following command.
```bash
go build --tags "<FEATURE>"
```
For available features see the extension list.
When using multiple build tags, all the different tags should be space delimted.
Example:
```bash
go build --tags "icu json1 fts5 secure_delete"
```
### Feature / Extension List
| Extension | Build Tag | Description |
|-----------|-----------|-------------|
| Additional Statistics | sqlite_stat4 | This option adds additional logic to the ANALYZE command and to the query planner that can help SQLite to chose a better query plan under certain situations. The ANALYZE command is enhanced to collect histogram data from all columns of every index and store that data in the sqlite_stat4 table.<br><br>The query planner will then use the histogram data to help it make better index choices. The downside of this compile-time option is that it violates the query planner stability guarantee making it more difficult to ensure consistent performance in mass-produced applications.<br><br>SQLITE_ENABLE_STAT4 is an enhancement of SQLITE_ENABLE_STAT3. STAT3 only recorded histogram data for the left-most column of each index whereas the STAT4 enhancement records histogram data from all columns of each index.<br><br>The SQLITE_ENABLE_STAT3 compile-time option is a no-op and is ignored if the SQLITE_ENABLE_STAT4 compile-time option is used |
| Allow URI Authority | sqlite_allow_uri_authority | URI filenames normally throws an error if the authority section is not either empty or "localhost".<br><br>However, if SQLite is compiled with the SQLITE_ALLOW_URI_AUTHORITY compile-time option, then the URI is converted into a Uniform Naming Convention (UNC) filename and passed down to the underlying operating system that way |
| App Armor | sqlite_app_armor | When defined, this C-preprocessor macro activates extra code that attempts to detect misuse of the SQLite API, such as passing in NULL pointers to required parameters or using objects after they have been destroyed. <br><br>App Armor is not available under `Windows`. |
| Disable Load Extensions | sqlite_omit_load_extension | Loading of external extensions is enabled by default.<br><br>To disable extension loading add the build tag `sqlite_omit_load_extension`. |
| Foreign Keys | sqlite_foreign_keys | This macro determines whether enforcement of foreign key constraints is enabled or disabled by default for new database connections.<br><br>Each database connection can always turn enforcement of foreign key constraints on and off and run-time using the foreign_keys pragma.<br><br>Enforcement of foreign key constraints is normally off by default, but if this compile-time parameter is set to 1, enforcement of foreign key constraints will be on by default |
| Full Auto Vacuum | sqlite_vacuum_full | Set the default auto vacuum to full |
| Incremental Auto Vacuum | sqlite_vacuum_incr | Set the default auto vacuum to incremental |
| Full Text Search Engine | sqlite_fts5 | When this option is defined in the amalgamation, versions 5 of the full-text search engine (fts5) is added to the build automatically |
| International Components for Unicode | sqlite_icu | This option causes the International Components for Unicode or "ICU" extension to SQLite to be added to the build |
| Introspect PRAGMAS | sqlite_introspect | This option adds some extra PRAGMA statements. <ul><li>PRAGMA function_list</li><li>PRAGMA module_list</li><li>PRAGMA pragma_list</li></ul> |
| JSON SQL Functions | sqlite_json | When this option is defined in the amalgamation, the JSON SQL functions are added to the build automatically |
| Pre Update Hook | sqlite_preupdate_hook | Registers a callback function that is invoked prior to each INSERT, UPDATE, and DELETE operation on a database table. |
| Secure Delete | sqlite_secure_delete | This compile-time option changes the default setting of the secure_delete pragma.<br><br>When this option is not used, secure_delete defaults to off. When this option is present, secure_delete defaults to on.<br><br>The secure_delete setting causes deleted content to be overwritten with zeros. There is a small performance penalty since additional I/O must occur.<br><br>On the other hand, secure_delete can prevent fragments of sensitive information from lingering in unused parts of the database file after it has been deleted. See the documentation on the secure_delete pragma for additional information |
| Secure Delete (FAST) | sqlite_secure_delete_fast | For more information see [PRAGMA secure_delete](https://www.sqlite.org/pragma.html#pragma_secure_delete) |
| Tracing / Debug | sqlite_trace | Activate trace functions |
| User Authentication | sqlite_userauth | SQLite User Authentication see [User Authentication](#user-authentication) for more information. |
# Compilation
This package requires `CGO_ENABLED=1` ennvironment variable if not set by default, and the presence of the `gcc` compiler.
If you need to add additional CFLAGS or LDFLAGS to the build command, and do not want to modify this package. Then this can be achieved by using the `CGO_CFLAGS` and `CGO_LDFLAGS` environment variables.
## Android
This package can be compiled for android.
Compile with:
```bash
go build --tags "android"
```
For more information see [#201](https://github.com/mattn/go-sqlite3/issues/201)
# ARM
To compile for `ARM` use the following environment.
```bash
env CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ \
CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 \
go build -v
```
Additional information:
- [#242](https://github.com/mattn/go-sqlite3/issues/242)
- [#504](https://github.com/mattn/go-sqlite3/issues/504)
# Cross Compile
This library can be cross-compiled.
In some cases you are required to the `CC` environment variable with the cross compiler.
## Cross Compiling from MAC OSX
The simplest way to cross compile from OSX is to use [xgo](https://github.com/karalabe/xgo).
Steps:
- Install [xgo](https://github.com/karalabe/xgo) (`go get github.com/karalabe/xgo`).
- Ensure that your project is within your `GOPATH`.
- Run `xgo local/path/to/project`.
Please refer to the project's [README](https://github.com/karalabe/xgo/blob/master/README.md) for further information.
# Google Cloud Platform
Building on GCP is not possible because Google Cloud Platform does not allow `gcc` to be executed.
Please work only with compiled final binaries.
## Linux
To compile this package on Linux you must install the development tools for your linux distribution.
To compile under linux use the build tag `linux`.
```bash
go build --tags "linux"
```
If you wish to link directly to libsqlite3 then you can use the `libsqlite3` build tag.
```
go build --tags "libsqlite3 linux"
```
### Alpine
When building in an `alpine` container run the following command before building.
```
apk add --update gcc musl-dev
```
### Fedora
```bash
sudo yum groupinstall "Development Tools" "Development Libraries"
```
### Ubuntu
```bash
sudo apt-get install build-essential
```
## Mac OSX
OSX should have all the tools present to compile this package, if not install XCode this will add all the developers tools.
Required dependency
```bash
brew install sqlite3
```
For OSX there is an additional package install which is required if you wish to build the `icu` extension.
This additional package can be installed with `homebrew`.
```bash
brew upgrade icu4c
```
To compile for Mac OSX.
```bash
go build --tags "darwin"
```
If you wish to link directly to libsqlite3 then you can use the `libsqlite3` build tag.
```
go build --tags "libsqlite3 darwin"
```
Additional information:
- [#206](https://github.com/mattn/go-sqlite3/issues/206)
- [#404](https://github.com/mattn/go-sqlite3/issues/404)
## Windows
To compile this package on Windows OS you must have the `gcc` compiler installed.
1) Install a Windows `gcc` toolchain.
2) Add the `bin` folders to the Windows path if the installer did not do this by default.
3) Open a terminal for the TDM-GCC toolchain, can be found in the Windows Start menu.
4) Navigate to your project folder and run the `go build ...` command for this package.
For example the TDM-GCC Toolchain can be found [here](https://sourceforge.net/projects/tdm-gcc/).
## Errors
- Compile error: `can not be used when making a shared object; recompile with -fPIC`
When receiving a compile time error referencing recompile with `-FPIC` then you
are probably using a hardend system.
You can compile the library on a hardend system with the following command.
```bash
go build -ldflags '-extldflags=-fno-PIC'
```
More details see [#120](https://github.com/mattn/go-sqlite3/issues/120)
- Can't build go-sqlite3 on windows 64bit.
> Probably, you are using go 1.0, go1.0 has a problem when it comes to compiling/linking on windows 64bit.
> See: [#27](https://github.com/mattn/go-sqlite3/issues/27)
- `go get github.com/mattn/go-sqlite3` throws compilation error.
`gcc` throws: `internal compiler error`
Remove the download repository from your disk and try re-install with:
```bash
go install github.com/mattn/go-sqlite3
```
# User Authentication
This package supports the SQLite User Authentication module.
## Compile
To use the User authentication module the package has to be compiled with the tag `sqlite_userauth`. See [Features](#features).
## Usage
### Create protected database
To create a database protected by user authentication provide the following argument to the connection string `_auth`.
This will enable user authentication within the database. This option however requires two additional arguments:
- `_auth_user`
- `_auth_pass`
When `_auth` is present on the connection string user authentication will be enabled and the provided user will be created
as an `admin` user. After initial creation, the parameter `_auth` has no effect anymore and can be omitted from the connection string.
Example connection string:
Create an user authentication database with user `admin` and password `admin`.
`file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin`
Create an user authentication database with user `admin` and password `admin` and use `SHA1` for the password encoding.
`file:test.s3db?_auth&_auth_user=admin&_auth_pass=admin&_auth_crypt=sha1`
### Password Encoding
The passwords within the user authentication module of SQLite are encoded with the SQLite function `sqlite_cryp`.
This function uses a ceasar-cypher which is quite insecure.
This library provides several additional password encoders which can be configured through the connection string.
The password cypher can be configured with the key `_auth_crypt`. And if the configured password encoder also requires an
salt this can be configured with `_auth_salt`.
#### Available Encoders
- SHA1
- SSHA1 (Salted SHA1)
- SHA256
- SSHA256 (salted SHA256)
- SHA384
- SSHA384 (salted SHA384)
- SHA512
- SSHA512 (salted SHA512)
### Restrictions
Operations on the database regarding to user management can only be preformed by an administrator user.
### Support
The user authentication supports two kinds of users
- administrators
- regular users
### User Management
User management can be done by directly using the `*SQLiteConn` or by SQL.
#### SQL
The following sql functions are available for user management.
| Function | Arguments | Description |
|----------|-----------|-------------|
| `authenticate` | username `string`, password `string` | Will authenticate an user, this is done by the connection; and should not be used manually. |
| `auth_user_add` | username `string`, password `string`, admin `int` | This function will add an user to the database.<br>if the database is not protected by user authentication it will enable it. Argument `admin` is an integer identifying if the added user should be an administrator. Only Administrators can add administrators. |
| `auth_user_change` | username `string`, password `string`, admin `int` | Function to modify an user. Users can change their own password, but only an administrator can change the administrator flag. |
| `authUserDelete` | username `string` | Delete an user from the database. Can only be used by an administrator. The current logged in administrator cannot be deleted. This is to make sure their is always an administrator remaining. |
These functions will return an integer.
- 0 (SQLITE_OK)
- 23 (SQLITE_AUTH) Failed to perform due to authentication or insufficient privileges
##### Examples
```sql
// Autheticate user
// Create Admin User
SELECT auth_user_add('admin2', 'admin2', 1);
// Change password for user
SELECT auth_user_change('user', 'userpassword', 0);
// Delete user
SELECT user_delete('user');
```
#### *SQLiteConn
The following functions are available for User authentication from the `*SQLiteConn`.
| Function | Description |
|----------|-------------|
| `Authenticate(username, password string) error` | Authenticate user |
| `AuthUserAdd(username, password string, admin bool) error` | Add user |
| `AuthUserChange(username, password string, admin bool) error` | Modify user |
| `AuthUserDelete(username string) error` | Delete user |
### Attached database
When using attached databases. SQLite will use the authentication from the `main` database for the attached database(s).
# Extensions
If you want your own extension to be listed here or you want to add a reference to an extension; please submit an Issue for this.
## Spatialite
Spatialite is available as an extension to SQLite, and can be used in combination with this repository.
For an example see [shaxbee/go-spatialite](https://github.com/shaxbee/go-spatialite).
## extension-functions.c from SQLite3 Contrib
extension-functions.c is available as an extension to SQLite, and provides the following functions:
- Math: acos, asin, atan, atn2, atan2, acosh, asinh, atanh, difference, degrees, radians, cos, sin, tan, cot, cosh, sinh, tanh, coth, exp, log, log10, power, sign, sqrt, square, ceil, floor, pi.
- String: replicate, charindex, leftstr, rightstr, ltrim, rtrim, trim, replace, reverse, proper, padl, padr, padc, strfilter.
- Aggregate: stdev, variance, mode, median, lower_quartile, upper_quartile
For an example see [dinedal/go-sqlite3-extension-functions](https://github.com/dinedal/go-sqlite3-extension-functions).
# FAQ
- Getting insert error while query is opened.
> You can pass some arguments into the connection string, for example, a URI.
> See: [#39](https://github.com/mattn/go-sqlite3/issues/39)
- Do you want to cross compile? mingw on Linux or Mac?
> See: [#106](https://github.com/mattn/go-sqlite3/issues/106)
> See also: http://www.limitlessfx.com/cross-compile-golang-app-for-windows-from-linux.html
- Want to get time.Time with current locale
Use `_loc=auto` in SQLite3 filename schema like `file:foo.db?_loc=auto`.
- Can I use this in multiple routines concurrently?
Yes for readonly. But, No for writable. See [#50](https://github.com/mattn/go-sqlite3/issues/50), [#51](https://github.com/mattn/go-sqlite3/issues/51), [#209](https://github.com/mattn/go-sqlite3/issues/209), [#274](https://github.com/mattn/go-sqlite3/issues/274).
- Why I'm getting `no such table` error?
Why is it racy if I use a `sql.Open("sqlite3", ":memory:")` database?
Each connection to `":memory:"` opens a brand new in-memory sql database, so if
the stdlib's sql engine happens to open another connection and you've only
specified `":memory:"`, that connection will see a brand new database. A
workaround is to use `"file::memory:?cache=shared"` (or `"file:foobar?mode=memory&cache=shared"`). Every
connection to this string will point to the same in-memory database.
Note that if the last database connection in the pool closes, the in-memory database is deleted. Make sure the [max idle connection limit](https://golang.org/pkg/database/sql/#DB.SetMaxIdleConns) is > 0, and the [connection lifetime](https://golang.org/pkg/database/sql/#DB.SetConnMaxLifetime) is infinite.
For more information see
* [#204](https://github.com/mattn/go-sqlite3/issues/204)
* [#511](https://github.com/mattn/go-sqlite3/issues/511)
* https://www.sqlite.org/sharedcache.html#shared_cache_and_in_memory_databases
* https://www.sqlite.org/inmemorydb.html#sharedmemdb
- Reading from database with large amount of goroutines fails on OSX.
OS X limits OS-wide to not have more than 1000 files open simultaneously by default.
For more information see [#289](https://github.com/mattn/go-sqlite3/issues/289)
- Trying to execute a `.` (dot) command throws an error.
Error: `Error: near ".": syntax error`
Dot command are part of SQLite3 CLI not of this library.
You need to implement the feature or call the sqlite3 cli.
More information see [#305](https://github.com/mattn/go-sqlite3/issues/305)
- Error: `database is locked`
When you get a database is locked. Please use the following options.
Add to DSN: `cache=shared`
Example:
```go
db, err := sql.Open("sqlite3", "file:locked.sqlite?cache=shared")
```
Second please set the database connections of the SQL package to 1.
```go
db.SetMaxOpenConns(1)
```
More information see [#209](https://github.com/mattn/go-sqlite3/issues/209)
## Contributors
### Code Contributors
This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)].
<a href="https://github.com/mattn/go-sqlite3/graphs/contributors"><img src="https://opencollective.com/mattn-go-sqlite3/contributors.svg?width=890&button=false" /></a>
### Financial Contributors
Become a financial contributor and help us sustain our community. [[Contribute](https://opencollective.com/mattn-go-sqlite3/contribute)]
#### Individuals
<a href="https://opencollective.com/mattn-go-sqlite3"><img src="https://opencollective.com/mattn-go-sqlite3/individuals.svg?width=890"></a>
#### Organizations
Support this project with your organization. Your logo will show up here with a link to your website. [[Contribute](https://opencollective.com/mattn-go-sqlite3/contribute)]
<a href="https://opencollective.com/mattn-go-sqlite3/organization/0/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/0/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/1/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/1/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/2/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/2/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/3/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/3/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/4/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/4/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/5/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/5/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/6/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/6/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/7/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/7/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/8/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/8/avatar.svg"></a>
<a href="https://opencollective.com/mattn-go-sqlite3/organization/9/website"><img src="https://opencollective.com/mattn-go-sqlite3/organization/9/avatar.svg"></a>
# License
MIT: http://mattn.mit-license.org/2018
sqlite3-binding.c, sqlite3-binding.h, sqlite3ext.h
The -binding suffix was added to avoid build failures under gccgo.
In this repository, those files are an amalgamation of code that was copied from SQLite3. The license of that code is the same as the license of SQLite3.
# Author
Yasuhiro Matsumoto (a.k.a mattn)
G.J.R. Timmer

85
vendor/github.com/mattn/go-sqlite3/backup.go generated vendored Normal file
View File

@@ -0,0 +1,85 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
*/
import "C"
import (
"runtime"
"unsafe"
)
// SQLiteBackup implement interface of Backup.
type SQLiteBackup struct {
b *C.sqlite3_backup
}
// Backup make backup from src to dest.
func (destConn *SQLiteConn) Backup(dest string, srcConn *SQLiteConn, src string) (*SQLiteBackup, error) {
destptr := C.CString(dest)
defer C.free(unsafe.Pointer(destptr))
srcptr := C.CString(src)
defer C.free(unsafe.Pointer(srcptr))
if b := C.sqlite3_backup_init(destConn.db, destptr, srcConn.db, srcptr); b != nil {
bb := &SQLiteBackup{b: b}
runtime.SetFinalizer(bb, (*SQLiteBackup).Finish)
return bb, nil
}
return nil, destConn.lastError()
}
// Step to backs up for one step. Calls the underlying `sqlite3_backup_step`
// function. This function returns a boolean indicating if the backup is done
// and an error signalling any other error. Done is returned if the underlying
// C function returns SQLITE_DONE (Code 101)
func (b *SQLiteBackup) Step(p int) (bool, error) {
ret := C.sqlite3_backup_step(b.b, C.int(p))
if ret == C.SQLITE_DONE {
return true, nil
} else if ret != 0 && ret != C.SQLITE_LOCKED && ret != C.SQLITE_BUSY {
return false, Error{Code: ErrNo(ret)}
}
return false, nil
}
// Remaining return whether have the rest for backup.
func (b *SQLiteBackup) Remaining() int {
return int(C.sqlite3_backup_remaining(b.b))
}
// PageCount return count of pages.
func (b *SQLiteBackup) PageCount() int {
return int(C.sqlite3_backup_pagecount(b.b))
}
// Finish close backup.
func (b *SQLiteBackup) Finish() error {
return b.Close()
}
// Close close backup.
func (b *SQLiteBackup) Close() error {
ret := C.sqlite3_backup_finish(b.b)
// sqlite3_backup_finish() never fails, it just returns the
// error code from previous operations, so clean up before
// checking and returning an error
b.b = nil
runtime.SetFinalizer(b, nil)
if ret != 0 {
return Error{Code: ErrNo(ret)}
}
return nil
}

392
vendor/github.com/mattn/go-sqlite3/callback.go generated vendored Normal file
View File

@@ -0,0 +1,392 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
// You can't export a Go function to C and have definitions in the C
// preamble in the same file, so we have to have callbackTrampoline in
// its own file. Because we need a separate file anyway, the support
// code for SQLite custom functions is in here.
/*
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
void _sqlite3_result_text(sqlite3_context* ctx, const char* s);
void _sqlite3_result_blob(sqlite3_context* ctx, const void* b, int l);
*/
import "C"
import (
"errors"
"fmt"
"math"
"reflect"
"sync"
"unsafe"
)
//export callbackTrampoline
func callbackTrampoline(ctx *C.sqlite3_context, argc int, argv **C.sqlite3_value) {
args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:argc:argc]
fi := lookupHandle(C.sqlite3_user_data(ctx)).(*functionInfo)
fi.Call(ctx, args)
}
//export stepTrampoline
func stepTrampoline(ctx *C.sqlite3_context, argc C.int, argv **C.sqlite3_value) {
args := (*[(math.MaxInt32 - 1) / unsafe.Sizeof((*C.sqlite3_value)(nil))]*C.sqlite3_value)(unsafe.Pointer(argv))[:int(argc):int(argc)]
ai := lookupHandle(C.sqlite3_user_data(ctx)).(*aggInfo)
ai.Step(ctx, args)
}
//export doneTrampoline
func doneTrampoline(ctx *C.sqlite3_context) {
ai := lookupHandle(C.sqlite3_user_data(ctx)).(*aggInfo)
ai.Done(ctx)
}
//export compareTrampoline
func compareTrampoline(handlePtr unsafe.Pointer, la C.int, a *C.char, lb C.int, b *C.char) C.int {
cmp := lookupHandle(handlePtr).(func(string, string) int)
return C.int(cmp(C.GoStringN(a, la), C.GoStringN(b, lb)))
}
//export commitHookTrampoline
func commitHookTrampoline(handle unsafe.Pointer) int {
callback := lookupHandle(handle).(func() int)
return callback()
}
//export rollbackHookTrampoline
func rollbackHookTrampoline(handle unsafe.Pointer) {
callback := lookupHandle(handle).(func())
callback()
}
//export updateHookTrampoline
func updateHookTrampoline(handle unsafe.Pointer, op int, db *C.char, table *C.char, rowid int64) {
callback := lookupHandle(handle).(func(int, string, string, int64))
callback(op, C.GoString(db), C.GoString(table), rowid)
}
//export authorizerTrampoline
func authorizerTrampoline(handle unsafe.Pointer, op int, arg1 *C.char, arg2 *C.char, arg3 *C.char) int {
callback := lookupHandle(handle).(func(int, string, string, string) int)
return callback(op, C.GoString(arg1), C.GoString(arg2), C.GoString(arg3))
}
//export preUpdateHookTrampoline
func preUpdateHookTrampoline(handle unsafe.Pointer, dbHandle uintptr, op int, db *C.char, table *C.char, oldrowid int64, newrowid int64) {
hval := lookupHandleVal(handle)
data := SQLitePreUpdateData{
Conn: hval.db,
Op: op,
DatabaseName: C.GoString(db),
TableName: C.GoString(table),
OldRowID: oldrowid,
NewRowID: newrowid,
}
callback := hval.val.(func(SQLitePreUpdateData))
callback(data)
}
// Use handles to avoid passing Go pointers to C.
type handleVal struct {
db *SQLiteConn
val interface{}
}
var handleLock sync.Mutex
var handleVals = make(map[unsafe.Pointer]handleVal)
func newHandle(db *SQLiteConn, v interface{}) unsafe.Pointer {
handleLock.Lock()
defer handleLock.Unlock()
val := handleVal{db: db, val: v}
var p unsafe.Pointer = C.malloc(C.size_t(1))
if p == nil {
panic("can't allocate 'cgo-pointer hack index pointer': ptr == nil")
}
handleVals[p] = val
return p
}
func lookupHandleVal(handle unsafe.Pointer) handleVal {
handleLock.Lock()
defer handleLock.Unlock()
return handleVals[handle]
}
func lookupHandle(handle unsafe.Pointer) interface{} {
return lookupHandleVal(handle).val
}
func deleteHandles(db *SQLiteConn) {
handleLock.Lock()
defer handleLock.Unlock()
for handle, val := range handleVals {
if val.db == db {
delete(handleVals, handle)
C.free(handle)
}
}
}
// This is only here so that tests can refer to it.
type callbackArgRaw C.sqlite3_value
type callbackArgConverter func(*C.sqlite3_value) (reflect.Value, error)
type callbackArgCast struct {
f callbackArgConverter
typ reflect.Type
}
func (c callbackArgCast) Run(v *C.sqlite3_value) (reflect.Value, error) {
val, err := c.f(v)
if err != nil {
return reflect.Value{}, err
}
if !val.Type().ConvertibleTo(c.typ) {
return reflect.Value{}, fmt.Errorf("cannot convert %s to %s", val.Type(), c.typ)
}
return val.Convert(c.typ), nil
}
func callbackArgInt64(v *C.sqlite3_value) (reflect.Value, error) {
if C.sqlite3_value_type(v) != C.SQLITE_INTEGER {
return reflect.Value{}, fmt.Errorf("argument must be an INTEGER")
}
return reflect.ValueOf(int64(C.sqlite3_value_int64(v))), nil
}
func callbackArgBool(v *C.sqlite3_value) (reflect.Value, error) {
if C.sqlite3_value_type(v) != C.SQLITE_INTEGER {
return reflect.Value{}, fmt.Errorf("argument must be an INTEGER")
}
i := int64(C.sqlite3_value_int64(v))
val := false
if i != 0 {
val = true
}
return reflect.ValueOf(val), nil
}
func callbackArgFloat64(v *C.sqlite3_value) (reflect.Value, error) {
if C.sqlite3_value_type(v) != C.SQLITE_FLOAT {
return reflect.Value{}, fmt.Errorf("argument must be a FLOAT")
}
return reflect.ValueOf(float64(C.sqlite3_value_double(v))), nil
}
func callbackArgBytes(v *C.sqlite3_value) (reflect.Value, error) {
switch C.sqlite3_value_type(v) {
case C.SQLITE_BLOB:
l := C.sqlite3_value_bytes(v)
p := C.sqlite3_value_blob(v)
return reflect.ValueOf(C.GoBytes(p, l)), nil
case C.SQLITE_TEXT:
l := C.sqlite3_value_bytes(v)
c := unsafe.Pointer(C.sqlite3_value_text(v))
return reflect.ValueOf(C.GoBytes(c, l)), nil
default:
return reflect.Value{}, fmt.Errorf("argument must be BLOB or TEXT")
}
}
func callbackArgString(v *C.sqlite3_value) (reflect.Value, error) {
switch C.sqlite3_value_type(v) {
case C.SQLITE_BLOB:
l := C.sqlite3_value_bytes(v)
p := (*C.char)(C.sqlite3_value_blob(v))
return reflect.ValueOf(C.GoStringN(p, l)), nil
case C.SQLITE_TEXT:
c := (*C.char)(unsafe.Pointer(C.sqlite3_value_text(v)))
return reflect.ValueOf(C.GoString(c)), nil
default:
return reflect.Value{}, fmt.Errorf("argument must be BLOB or TEXT")
}
}
func callbackArgGeneric(v *C.sqlite3_value) (reflect.Value, error) {
switch C.sqlite3_value_type(v) {
case C.SQLITE_INTEGER:
return callbackArgInt64(v)
case C.SQLITE_FLOAT:
return callbackArgFloat64(v)
case C.SQLITE_TEXT:
return callbackArgString(v)
case C.SQLITE_BLOB:
return callbackArgBytes(v)
case C.SQLITE_NULL:
// Interpret NULL as a nil byte slice.
var ret []byte
return reflect.ValueOf(ret), nil
default:
panic("unreachable")
}
}
func callbackArg(typ reflect.Type) (callbackArgConverter, error) {
switch typ.Kind() {
case reflect.Interface:
if typ.NumMethod() != 0 {
return nil, errors.New("the only supported interface type is interface{}")
}
return callbackArgGeneric, nil
case reflect.Slice:
if typ.Elem().Kind() != reflect.Uint8 {
return nil, errors.New("the only supported slice type is []byte")
}
return callbackArgBytes, nil
case reflect.String:
return callbackArgString, nil
case reflect.Bool:
return callbackArgBool, nil
case reflect.Int64:
return callbackArgInt64, nil
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Int, reflect.Uint:
c := callbackArgCast{callbackArgInt64, typ}
return c.Run, nil
case reflect.Float64:
return callbackArgFloat64, nil
case reflect.Float32:
c := callbackArgCast{callbackArgFloat64, typ}
return c.Run, nil
default:
return nil, fmt.Errorf("don't know how to convert to %s", typ)
}
}
func callbackConvertArgs(argv []*C.sqlite3_value, converters []callbackArgConverter, variadic callbackArgConverter) ([]reflect.Value, error) {
var args []reflect.Value
if len(argv) < len(converters) {
return nil, fmt.Errorf("function requires at least %d arguments", len(converters))
}
for i, arg := range argv[:len(converters)] {
v, err := converters[i](arg)
if err != nil {
return nil, err
}
args = append(args, v)
}
if variadic != nil {
for _, arg := range argv[len(converters):] {
v, err := variadic(arg)
if err != nil {
return nil, err
}
args = append(args, v)
}
}
return args, nil
}
type callbackRetConverter func(*C.sqlite3_context, reflect.Value) error
func callbackRetInteger(ctx *C.sqlite3_context, v reflect.Value) error {
switch v.Type().Kind() {
case reflect.Int64:
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Int, reflect.Uint:
v = v.Convert(reflect.TypeOf(int64(0)))
case reflect.Bool:
b := v.Interface().(bool)
if b {
v = reflect.ValueOf(int64(1))
} else {
v = reflect.ValueOf(int64(0))
}
default:
return fmt.Errorf("cannot convert %s to INTEGER", v.Type())
}
C.sqlite3_result_int64(ctx, C.sqlite3_int64(v.Interface().(int64)))
return nil
}
func callbackRetFloat(ctx *C.sqlite3_context, v reflect.Value) error {
switch v.Type().Kind() {
case reflect.Float64:
case reflect.Float32:
v = v.Convert(reflect.TypeOf(float64(0)))
default:
return fmt.Errorf("cannot convert %s to FLOAT", v.Type())
}
C.sqlite3_result_double(ctx, C.double(v.Interface().(float64)))
return nil
}
func callbackRetBlob(ctx *C.sqlite3_context, v reflect.Value) error {
if v.Type().Kind() != reflect.Slice || v.Type().Elem().Kind() != reflect.Uint8 {
return fmt.Errorf("cannot convert %s to BLOB", v.Type())
}
i := v.Interface()
if i == nil || len(i.([]byte)) == 0 {
C.sqlite3_result_null(ctx)
} else {
bs := i.([]byte)
C._sqlite3_result_blob(ctx, unsafe.Pointer(&bs[0]), C.int(len(bs)))
}
return nil
}
func callbackRetText(ctx *C.sqlite3_context, v reflect.Value) error {
if v.Type().Kind() != reflect.String {
return fmt.Errorf("cannot convert %s to TEXT", v.Type())
}
C._sqlite3_result_text(ctx, C.CString(v.Interface().(string)))
return nil
}
func callbackRetNil(ctx *C.sqlite3_context, v reflect.Value) error {
return nil
}
func callbackRet(typ reflect.Type) (callbackRetConverter, error) {
switch typ.Kind() {
case reflect.Interface:
errorInterface := reflect.TypeOf((*error)(nil)).Elem()
if typ.Implements(errorInterface) {
return callbackRetNil, nil
}
fallthrough
case reflect.Slice:
if typ.Elem().Kind() != reflect.Uint8 {
return nil, errors.New("the only supported slice type is []byte")
}
return callbackRetBlob, nil
case reflect.String:
return callbackRetText, nil
case reflect.Bool, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Int, reflect.Uint:
return callbackRetInteger, nil
case reflect.Float32, reflect.Float64:
return callbackRetFloat, nil
default:
return nil, fmt.Errorf("don't know how to convert to %s", typ)
}
}
func callbackError(ctx *C.sqlite3_context, err error) {
cstr := C.CString(err.Error())
defer C.free(unsafe.Pointer(cstr))
C.sqlite3_result_error(ctx, cstr, C.int(-1))
}
// Test support code. Tests are not allowed to import "C", so we can't
// declare any functions that use C.sqlite3_value.
func callbackSyntheticForTests(v reflect.Value, err error) callbackArgConverter {
return func(*C.sqlite3_value) (reflect.Value, error) {
return v, err
}
}

299
vendor/github.com/mattn/go-sqlite3/convert.go generated vendored Normal file
View File

@@ -0,0 +1,299 @@
// Extracted from Go database/sql source code
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Type conversions for Scan.
package sqlite3
import (
"database/sql"
"database/sql/driver"
"errors"
"fmt"
"reflect"
"strconv"
"time"
)
var errNilPtr = errors.New("destination pointer is nil") // embedded in descriptive error
// convertAssign copies to dest the value in src, converting it if possible.
// An error is returned if the copy would result in loss of information.
// dest should be a pointer type.
func convertAssign(dest, src interface{}) error {
// Common cases, without reflect.
switch s := src.(type) {
case string:
switch d := dest.(type) {
case *string:
if d == nil {
return errNilPtr
}
*d = s
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = []byte(s)
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = append((*d)[:0], s...)
return nil
}
case []byte:
switch d := dest.(type) {
case *string:
if d == nil {
return errNilPtr
}
*d = string(s)
return nil
case *interface{}:
if d == nil {
return errNilPtr
}
*d = cloneBytes(s)
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = cloneBytes(s)
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = s
return nil
}
case time.Time:
switch d := dest.(type) {
case *time.Time:
*d = s
return nil
case *string:
*d = s.Format(time.RFC3339Nano)
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = []byte(s.Format(time.RFC3339Nano))
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = s.AppendFormat((*d)[:0], time.RFC3339Nano)
return nil
}
case nil:
switch d := dest.(type) {
case *interface{}:
if d == nil {
return errNilPtr
}
*d = nil
return nil
case *[]byte:
if d == nil {
return errNilPtr
}
*d = nil
return nil
case *sql.RawBytes:
if d == nil {
return errNilPtr
}
*d = nil
return nil
}
}
var sv reflect.Value
switch d := dest.(type) {
case *string:
sv = reflect.ValueOf(src)
switch sv.Kind() {
case reflect.Bool,
reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64,
reflect.Float32, reflect.Float64:
*d = asString(src)
return nil
}
case *[]byte:
sv = reflect.ValueOf(src)
if b, ok := asBytes(nil, sv); ok {
*d = b
return nil
}
case *sql.RawBytes:
sv = reflect.ValueOf(src)
if b, ok := asBytes([]byte(*d)[:0], sv); ok {
*d = sql.RawBytes(b)
return nil
}
case *bool:
bv, err := driver.Bool.ConvertValue(src)
if err == nil {
*d = bv.(bool)
}
return err
case *interface{}:
*d = src
return nil
}
if scanner, ok := dest.(sql.Scanner); ok {
return scanner.Scan(src)
}
dpv := reflect.ValueOf(dest)
if dpv.Kind() != reflect.Ptr {
return errors.New("destination not a pointer")
}
if dpv.IsNil() {
return errNilPtr
}
if !sv.IsValid() {
sv = reflect.ValueOf(src)
}
dv := reflect.Indirect(dpv)
if sv.IsValid() && sv.Type().AssignableTo(dv.Type()) {
switch b := src.(type) {
case []byte:
dv.Set(reflect.ValueOf(cloneBytes(b)))
default:
dv.Set(sv)
}
return nil
}
if dv.Kind() == sv.Kind() && sv.Type().ConvertibleTo(dv.Type()) {
dv.Set(sv.Convert(dv.Type()))
return nil
}
// The following conversions use a string value as an intermediate representation
// to convert between various numeric types.
//
// This also allows scanning into user defined types such as "type Int int64".
// For symmetry, also check for string destination types.
switch dv.Kind() {
case reflect.Ptr:
if src == nil {
dv.Set(reflect.Zero(dv.Type()))
return nil
}
dv.Set(reflect.New(dv.Type().Elem()))
return convertAssign(dv.Interface(), src)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
s := asString(src)
i64, err := strconv.ParseInt(s, 10, dv.Type().Bits())
if err != nil {
err = strconvErr(err)
return fmt.Errorf("converting driver.Value type %T (%q) to a %s: %v", src, s, dv.Kind(), err)
}
dv.SetInt(i64)
return nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
s := asString(src)
u64, err := strconv.ParseUint(s, 10, dv.Type().Bits())
if err != nil {
err = strconvErr(err)
return fmt.Errorf("converting driver.Value type %T (%q) to a %s: %v", src, s, dv.Kind(), err)
}
dv.SetUint(u64)
return nil
case reflect.Float32, reflect.Float64:
s := asString(src)
f64, err := strconv.ParseFloat(s, dv.Type().Bits())
if err != nil {
err = strconvErr(err)
return fmt.Errorf("converting driver.Value type %T (%q) to a %s: %v", src, s, dv.Kind(), err)
}
dv.SetFloat(f64)
return nil
case reflect.String:
switch v := src.(type) {
case string:
dv.SetString(v)
return nil
case []byte:
dv.SetString(string(v))
return nil
}
}
return fmt.Errorf("unsupported Scan, storing driver.Value type %T into type %T", src, dest)
}
func strconvErr(err error) error {
if ne, ok := err.(*strconv.NumError); ok {
return ne.Err
}
return err
}
func cloneBytes(b []byte) []byte {
if b == nil {
return nil
}
c := make([]byte, len(b))
copy(c, b)
return c
}
func asString(src interface{}) string {
switch v := src.(type) {
case string:
return v
case []byte:
return string(v)
}
rv := reflect.ValueOf(src)
switch rv.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return strconv.FormatInt(rv.Int(), 10)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return strconv.FormatUint(rv.Uint(), 10)
case reflect.Float64:
return strconv.FormatFloat(rv.Float(), 'g', -1, 64)
case reflect.Float32:
return strconv.FormatFloat(rv.Float(), 'g', -1, 32)
case reflect.Bool:
return strconv.FormatBool(rv.Bool())
}
return fmt.Sprintf("%v", src)
}
func asBytes(buf []byte, rv reflect.Value) (b []byte, ok bool) {
switch rv.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return strconv.AppendInt(buf, rv.Int(), 10), true
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return strconv.AppendUint(buf, rv.Uint(), 10), true
case reflect.Float32:
return strconv.AppendFloat(buf, rv.Float(), 'g', -1, 32), true
case reflect.Float64:
return strconv.AppendFloat(buf, rv.Float(), 'g', -1, 64), true
case reflect.Bool:
return strconv.AppendBool(buf, rv.Bool()), true
case reflect.String:
s := rv.String()
return append(buf, s...), true
}
return
}

135
vendor/github.com/mattn/go-sqlite3/doc.go generated vendored Normal file
View File

@@ -0,0 +1,135 @@
/*
Package sqlite3 provides interface to SQLite3 databases.
This works as a driver for database/sql.
Installation
go get github.com/mattn/go-sqlite3
Supported Types
Currently, go-sqlite3 supports the following data types.
+------------------------------+
|go | sqlite3 |
|----------|-------------------|
|nil | null |
|int | integer |
|int64 | integer |
|float64 | float |
|bool | integer |
|[]byte | blob |
|string | text |
|time.Time | timestamp/datetime|
+------------------------------+
SQLite3 Extension
You can write your own extension module for sqlite3. For example, below is an
extension for a Regexp matcher operation.
#include <pcre.h>
#include <string.h>
#include <stdio.h>
#include <sqlite3ext.h>
SQLITE_EXTENSION_INIT1
static void regexp_func(sqlite3_context *context, int argc, sqlite3_value **argv) {
if (argc >= 2) {
const char *target = (const char *)sqlite3_value_text(argv[1]);
const char *pattern = (const char *)sqlite3_value_text(argv[0]);
const char* errstr = NULL;
int erroff = 0;
int vec[500];
int n, rc;
pcre* re = pcre_compile(pattern, 0, &errstr, &erroff, NULL);
rc = pcre_exec(re, NULL, target, strlen(target), 0, 0, vec, 500);
if (rc <= 0) {
sqlite3_result_error(context, errstr, 0);
return;
}
sqlite3_result_int(context, 1);
}
}
#ifdef _WIN32
__declspec(dllexport)
#endif
int sqlite3_extension_init(sqlite3 *db, char **errmsg,
const sqlite3_api_routines *api) {
SQLITE_EXTENSION_INIT2(api);
return sqlite3_create_function(db, "regexp", 2, SQLITE_UTF8,
(void*)db, regexp_func, NULL, NULL);
}
It needs to be built as a so/dll shared library. And you need to register
the extension module like below.
sql.Register("sqlite3_with_extensions",
&sqlite3.SQLiteDriver{
Extensions: []string{
"sqlite3_mod_regexp",
},
})
Then, you can use this extension.
rows, err := db.Query("select text from mytable where name regexp '^golang'")
Connection Hook
You can hook and inject your code when the connection is established by setting
ConnectHook to get the SQLiteConn.
sql.Register("sqlite3_with_hook_example",
&sqlite3.SQLiteDriver{
ConnectHook: func(conn *sqlite3.SQLiteConn) error {
sqlite3conn = append(sqlite3conn, conn)
return nil
},
})
You can also use database/sql.Conn.Raw (Go >= 1.13):
conn, err := db.Conn(context.Background())
// if err != nil { ... }
defer conn.Close()
err = conn.Raw(func (driverConn interface{}) error {
sqliteConn := driverConn.(*sqlite3.SQLiteConn)
// ... use sqliteConn
})
// if err != nil { ... }
Go SQlite3 Extensions
If you want to register Go functions as SQLite extension functions
you can make a custom driver by calling RegisterFunction from
ConnectHook.
regex = func(re, s string) (bool, error) {
return regexp.MatchString(re, s)
}
sql.Register("sqlite3_extended",
&sqlite3.SQLiteDriver{
ConnectHook: func(conn *sqlite3.SQLiteConn) error {
return conn.RegisterFunc("regexp", regex, true)
},
})
You can then use the custom driver by passing its name to sql.Open.
var i int
conn, err := sql.Open("sqlite3_extended", "./foo.db")
if err != nil {
panic(err)
}
err = db.QueryRow(`SELECT regexp("foo.*", "seafood")`).Scan(&i)
if err != nil {
panic(err)
}
See the documentation of RegisterFunc for more details.
*/
package sqlite3

150
vendor/github.com/mattn/go-sqlite3/error.go generated vendored Normal file
View File

@@ -0,0 +1,150 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
*/
import "C"
import "syscall"
// ErrNo inherit errno.
type ErrNo int
// ErrNoMask is mask code.
const ErrNoMask C.int = 0xff
// ErrNoExtended is extended errno.
type ErrNoExtended int
// Error implement sqlite error code.
type Error struct {
Code ErrNo /* The error code returned by SQLite */
ExtendedCode ErrNoExtended /* The extended error code returned by SQLite */
SystemErrno syscall.Errno /* The system errno returned by the OS through SQLite, if applicable */
err string /* The error string returned by sqlite3_errmsg(),
this usually contains more specific details. */
}
// result codes from http://www.sqlite.org/c3ref/c_abort.html
var (
ErrError = ErrNo(1) /* SQL error or missing database */
ErrInternal = ErrNo(2) /* Internal logic error in SQLite */
ErrPerm = ErrNo(3) /* Access permission denied */
ErrAbort = ErrNo(4) /* Callback routine requested an abort */
ErrBusy = ErrNo(5) /* The database file is locked */
ErrLocked = ErrNo(6) /* A table in the database is locked */
ErrNomem = ErrNo(7) /* A malloc() failed */
ErrReadonly = ErrNo(8) /* Attempt to write a readonly database */
ErrInterrupt = ErrNo(9) /* Operation terminated by sqlite3_interrupt() */
ErrIoErr = ErrNo(10) /* Some kind of disk I/O error occurred */
ErrCorrupt = ErrNo(11) /* The database disk image is malformed */
ErrNotFound = ErrNo(12) /* Unknown opcode in sqlite3_file_control() */
ErrFull = ErrNo(13) /* Insertion failed because database is full */
ErrCantOpen = ErrNo(14) /* Unable to open the database file */
ErrProtocol = ErrNo(15) /* Database lock protocol error */
ErrEmpty = ErrNo(16) /* Database is empty */
ErrSchema = ErrNo(17) /* The database schema changed */
ErrTooBig = ErrNo(18) /* String or BLOB exceeds size limit */
ErrConstraint = ErrNo(19) /* Abort due to constraint violation */
ErrMismatch = ErrNo(20) /* Data type mismatch */
ErrMisuse = ErrNo(21) /* Library used incorrectly */
ErrNoLFS = ErrNo(22) /* Uses OS features not supported on host */
ErrAuth = ErrNo(23) /* Authorization denied */
ErrFormat = ErrNo(24) /* Auxiliary database format error */
ErrRange = ErrNo(25) /* 2nd parameter to sqlite3_bind out of range */
ErrNotADB = ErrNo(26) /* File opened that is not a database file */
ErrNotice = ErrNo(27) /* Notifications from sqlite3_log() */
ErrWarning = ErrNo(28) /* Warnings from sqlite3_log() */
)
// Error return error message from errno.
func (err ErrNo) Error() string {
return Error{Code: err}.Error()
}
// Extend return extended errno.
func (err ErrNo) Extend(by int) ErrNoExtended {
return ErrNoExtended(int(err) | (by << 8))
}
// Error return error message that is extended code.
func (err ErrNoExtended) Error() string {
return Error{Code: ErrNo(C.int(err) & ErrNoMask), ExtendedCode: err}.Error()
}
func (err Error) Error() string {
var str string
if err.err != "" {
str = err.err
} else {
str = C.GoString(C.sqlite3_errstr(C.int(err.Code)))
}
if err.SystemErrno != 0 {
str += ": " + err.SystemErrno.Error()
}
return str
}
// result codes from http://www.sqlite.org/c3ref/c_abort_rollback.html
var (
ErrIoErrRead = ErrIoErr.Extend(1)
ErrIoErrShortRead = ErrIoErr.Extend(2)
ErrIoErrWrite = ErrIoErr.Extend(3)
ErrIoErrFsync = ErrIoErr.Extend(4)
ErrIoErrDirFsync = ErrIoErr.Extend(5)
ErrIoErrTruncate = ErrIoErr.Extend(6)
ErrIoErrFstat = ErrIoErr.Extend(7)
ErrIoErrUnlock = ErrIoErr.Extend(8)
ErrIoErrRDlock = ErrIoErr.Extend(9)
ErrIoErrDelete = ErrIoErr.Extend(10)
ErrIoErrBlocked = ErrIoErr.Extend(11)
ErrIoErrNoMem = ErrIoErr.Extend(12)
ErrIoErrAccess = ErrIoErr.Extend(13)
ErrIoErrCheckReservedLock = ErrIoErr.Extend(14)
ErrIoErrLock = ErrIoErr.Extend(15)
ErrIoErrClose = ErrIoErr.Extend(16)
ErrIoErrDirClose = ErrIoErr.Extend(17)
ErrIoErrSHMOpen = ErrIoErr.Extend(18)
ErrIoErrSHMSize = ErrIoErr.Extend(19)
ErrIoErrSHMLock = ErrIoErr.Extend(20)
ErrIoErrSHMMap = ErrIoErr.Extend(21)
ErrIoErrSeek = ErrIoErr.Extend(22)
ErrIoErrDeleteNoent = ErrIoErr.Extend(23)
ErrIoErrMMap = ErrIoErr.Extend(24)
ErrIoErrGetTempPath = ErrIoErr.Extend(25)
ErrIoErrConvPath = ErrIoErr.Extend(26)
ErrLockedSharedCache = ErrLocked.Extend(1)
ErrBusyRecovery = ErrBusy.Extend(1)
ErrBusySnapshot = ErrBusy.Extend(2)
ErrCantOpenNoTempDir = ErrCantOpen.Extend(1)
ErrCantOpenIsDir = ErrCantOpen.Extend(2)
ErrCantOpenFullPath = ErrCantOpen.Extend(3)
ErrCantOpenConvPath = ErrCantOpen.Extend(4)
ErrCorruptVTab = ErrCorrupt.Extend(1)
ErrReadonlyRecovery = ErrReadonly.Extend(1)
ErrReadonlyCantLock = ErrReadonly.Extend(2)
ErrReadonlyRollback = ErrReadonly.Extend(3)
ErrReadonlyDbMoved = ErrReadonly.Extend(4)
ErrAbortRollback = ErrAbort.Extend(2)
ErrConstraintCheck = ErrConstraint.Extend(1)
ErrConstraintCommitHook = ErrConstraint.Extend(2)
ErrConstraintForeignKey = ErrConstraint.Extend(3)
ErrConstraintFunction = ErrConstraint.Extend(4)
ErrConstraintNotNull = ErrConstraint.Extend(5)
ErrConstraintPrimaryKey = ErrConstraint.Extend(6)
ErrConstraintTrigger = ErrConstraint.Extend(7)
ErrConstraintUnique = ErrConstraint.Extend(8)
ErrConstraintVTab = ErrConstraint.Extend(9)
ErrConstraintRowID = ErrConstraint.Extend(10)
ErrNoticeRecoverWAL = ErrNotice.Extend(1)
ErrNoticeRecoverRollback = ErrNotice.Extend(2)
ErrWarningAutoIndex = ErrWarning.Extend(1)
)

234589
vendor/github.com/mattn/go-sqlite3/sqlite3-binding.c generated vendored Normal file

File diff suppressed because it is too large Load Diff

12365
vendor/github.com/mattn/go-sqlite3/sqlite3-binding.h generated vendored Normal file

File diff suppressed because it is too large Load Diff

2183
vendor/github.com/mattn/go-sqlite3/sqlite3.go generated vendored Normal file

File diff suppressed because it is too large Load Diff

103
vendor/github.com/mattn/go-sqlite3/sqlite3_context.go generated vendored Normal file
View File

@@ -0,0 +1,103 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
// These wrappers are necessary because SQLITE_TRANSIENT
// is a pointer constant, and cgo doesn't translate them correctly.
static inline void my_result_text(sqlite3_context *ctx, char *p, int np) {
sqlite3_result_text(ctx, p, np, SQLITE_TRANSIENT);
}
static inline void my_result_blob(sqlite3_context *ctx, void *p, int np) {
sqlite3_result_blob(ctx, p, np, SQLITE_TRANSIENT);
}
*/
import "C"
import (
"math"
"reflect"
"unsafe"
)
const i64 = unsafe.Sizeof(int(0)) > 4
// SQLiteContext behave sqlite3_context
type SQLiteContext C.sqlite3_context
// ResultBool sets the result of an SQL function.
func (c *SQLiteContext) ResultBool(b bool) {
if b {
c.ResultInt(1)
} else {
c.ResultInt(0)
}
}
// ResultBlob sets the result of an SQL function.
// See: sqlite3_result_blob, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultBlob(b []byte) {
if i64 && len(b) > math.MaxInt32 {
C.sqlite3_result_error_toobig((*C.sqlite3_context)(c))
return
}
var p *byte
if len(b) > 0 {
p = &b[0]
}
C.my_result_blob((*C.sqlite3_context)(c), unsafe.Pointer(p), C.int(len(b)))
}
// ResultDouble sets the result of an SQL function.
// See: sqlite3_result_double, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultDouble(d float64) {
C.sqlite3_result_double((*C.sqlite3_context)(c), C.double(d))
}
// ResultInt sets the result of an SQL function.
// See: sqlite3_result_int, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultInt(i int) {
if i64 && (i > math.MaxInt32 || i < math.MinInt32) {
C.sqlite3_result_int64((*C.sqlite3_context)(c), C.sqlite3_int64(i))
} else {
C.sqlite3_result_int((*C.sqlite3_context)(c), C.int(i))
}
}
// ResultInt64 sets the result of an SQL function.
// See: sqlite3_result_int64, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultInt64(i int64) {
C.sqlite3_result_int64((*C.sqlite3_context)(c), C.sqlite3_int64(i))
}
// ResultNull sets the result of an SQL function.
// See: sqlite3_result_null, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultNull() {
C.sqlite3_result_null((*C.sqlite3_context)(c))
}
// ResultText sets the result of an SQL function.
// See: sqlite3_result_text, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultText(s string) {
h := (*reflect.StringHeader)(unsafe.Pointer(&s))
cs, l := (*C.char)(unsafe.Pointer(h.Data)), C.int(h.Len)
C.my_result_text((*C.sqlite3_context)(c), cs, l)
}
// ResultZeroblob sets the result of an SQL function.
// See: sqlite3_result_zeroblob, http://sqlite.org/c3ref/result_blob.html
func (c *SQLiteContext) ResultZeroblob(n int) {
C.sqlite3_result_zeroblob((*C.sqlite3_context)(c), C.int(n))
}

View File

@@ -0,0 +1,120 @@
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
package sqlite3
import (
"crypto/sha1"
"crypto/sha256"
"crypto/sha512"
)
// This file provides several different implementations for the
// default embedded sqlite_crypt function.
// This function is uses a caesar-cypher by default
// and is used within the UserAuthentication module to encode
// the password.
//
// The provided functions can be used as an overload to the sqlite_crypt
// function through the use of the RegisterFunc on the connection.
//
// Because the functions can serv a purpose to an end-user
// without using the UserAuthentication module
// the functions are default compiled in.
//
// From SQLITE3 - user-auth.txt
// The sqlite_user.pw field is encoded by a built-in SQL function
// "sqlite_crypt(X,Y)". The two arguments are both BLOBs. The first argument
// is the plaintext password supplied to the sqlite3_user_authenticate()
// interface. The second argument is the sqlite_user.pw value and is supplied
// so that the function can extract the "salt" used by the password encoder.
// The result of sqlite_crypt(X,Y) is another blob which is the value that
// ends up being stored in sqlite_user.pw. To verify credentials X supplied
// by the sqlite3_user_authenticate() routine, SQLite runs:
//
// sqlite_user.pw == sqlite_crypt(X, sqlite_user.pw)
//
// To compute an appropriate sqlite_user.pw value from a new or modified
// password X, sqlite_crypt(X,NULL) is run. A new random salt is selected
// when the second argument is NULL.
//
// The built-in version of of sqlite_crypt() uses a simple Caesar-cypher
// which prevents passwords from being revealed by searching the raw database
// for ASCII text, but is otherwise trivally broken. For better password
// security, the database should be encrypted using the SQLite Encryption
// Extension or similar technology. Or, the application can use the
// sqlite3_create_function() interface to provide an alternative
// implementation of sqlite_crypt() that computes a stronger password hash,
// perhaps using a cryptographic hash function like SHA1.
// CryptEncoderSHA1 encodes a password with SHA1
func CryptEncoderSHA1(pass []byte, hash interface{}) []byte {
h := sha1.Sum(pass)
return h[:]
}
// CryptEncoderSSHA1 encodes a password with SHA1 with the
// configured salt.
func CryptEncoderSSHA1(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha1.Sum(p)
return h[:]
}
}
// CryptEncoderSHA256 encodes a password with SHA256
func CryptEncoderSHA256(pass []byte, hash interface{}) []byte {
h := sha256.Sum256(pass)
return h[:]
}
// CryptEncoderSSHA256 encodes a password with SHA256
// with the configured salt
func CryptEncoderSSHA256(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha256.Sum256(p)
return h[:]
}
}
// CryptEncoderSHA384 encodes a password with SHA384
func CryptEncoderSHA384(pass []byte, hash interface{}) []byte {
h := sha512.Sum384(pass)
return h[:]
}
// CryptEncoderSSHA384 encodes a password with SHA384
// with the configured salt
func CryptEncoderSSHA384(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha512.Sum384(p)
return h[:]
}
}
// CryptEncoderSHA512 encodes a password with SHA512
func CryptEncoderSHA512(pass []byte, hash interface{}) []byte {
h := sha512.Sum512(pass)
return h[:]
}
// CryptEncoderSSHA512 encodes a password with SHA512
// with the configured salt
func CryptEncoderSSHA512(salt string) func(pass []byte, hash interface{}) []byte {
return func(pass []byte, hash interface{}) []byte {
s := []byte(salt)
p := append(pass, s...)
h := sha512.Sum512(p)
return h[:]
}
}
// EOF

70
vendor/github.com/mattn/go-sqlite3/sqlite3_go18.go generated vendored Normal file
View File

@@ -0,0 +1,70 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build cgo
// +build go1.8
package sqlite3
import (
"database/sql/driver"
"context"
)
// Ping implement Pinger.
func (c *SQLiteConn) Ping(ctx context.Context) error {
if c.db == nil {
// must be ErrBadConn for sql to close the database
return driver.ErrBadConn
}
return nil
}
// QueryContext implement QueryerContext.
func (c *SQLiteConn) QueryContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Rows, error) {
list := make([]namedValue, len(args))
for i, nv := range args {
list[i] = namedValue(nv)
}
return c.query(ctx, query, list)
}
// ExecContext implement ExecerContext.
func (c *SQLiteConn) ExecContext(ctx context.Context, query string, args []driver.NamedValue) (driver.Result, error) {
list := make([]namedValue, len(args))
for i, nv := range args {
list[i] = namedValue(nv)
}
return c.exec(ctx, query, list)
}
// PrepareContext implement ConnPrepareContext.
func (c *SQLiteConn) PrepareContext(ctx context.Context, query string) (driver.Stmt, error) {
return c.prepare(ctx, query)
}
// BeginTx implement ConnBeginTx.
func (c *SQLiteConn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
return c.begin(ctx)
}
// QueryContext implement QueryerContext.
func (s *SQLiteStmt) QueryContext(ctx context.Context, args []driver.NamedValue) (driver.Rows, error) {
list := make([]namedValue, len(args))
for i, nv := range args {
list[i] = namedValue(nv)
}
return s.query(ctx, list)
}
// ExecContext implement ExecerContext.
func (s *SQLiteStmt) ExecContext(ctx context.Context, args []driver.NamedValue) (driver.Result, error) {
list := make([]namedValue, len(args))
for i, nv := range args {
list[i] = namedValue(nv)
}
return s.exec(ctx, list)
}

View File

@@ -0,0 +1,19 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build libsqlite3
package sqlite3
/*
#cgo CFLAGS: -DUSE_LIBSQLITE3
#cgo linux LDFLAGS: -lsqlite3
#cgo darwin LDFLAGS: -L/usr/local/opt/sqlite/lib -lsqlite3
#cgo darwin CFLAGS: -I/usr/local/opt/sqlite/include
#cgo openbsd LDFLAGS: -lsqlite3
#cgo solaris LDFLAGS: -lsqlite3
#cgo windows LDFLAGS: -lsqlite3
*/
import "C"

View File

@@ -0,0 +1,84 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !sqlite_omit_load_extension
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
*/
import "C"
import (
"errors"
"unsafe"
)
func (c *SQLiteConn) loadExtensions(extensions []string) error {
rv := C.sqlite3_enable_load_extension(c.db, 1)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
}
for _, extension := range extensions {
if err := c.loadExtension(extension, nil); err != nil {
C.sqlite3_enable_load_extension(c.db, 0)
return err
}
}
rv = C.sqlite3_enable_load_extension(c.db, 0)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
}
return nil
}
// LoadExtension load the sqlite3 extension.
func (c *SQLiteConn) LoadExtension(lib string, entry string) error {
rv := C.sqlite3_enable_load_extension(c.db, 1)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
}
if err := c.loadExtension(lib, &entry); err != nil {
C.sqlite3_enable_load_extension(c.db, 0)
return err
}
rv = C.sqlite3_enable_load_extension(c.db, 0)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(C.sqlite3_errmsg(c.db)))
}
return nil
}
func (c *SQLiteConn) loadExtension(lib string, entry *string) error {
clib := C.CString(lib)
defer C.free(unsafe.Pointer(clib))
var centry *C.char
if entry != nil {
centry = C.CString(*entry)
defer C.free(unsafe.Pointer(centry))
}
var errMsg *C.char
defer C.sqlite3_free(unsafe.Pointer(errMsg))
rv := C.sqlite3_load_extension(c.db, clib, centry, &errMsg)
if rv != C.SQLITE_OK {
return errors.New(C.GoString(errMsg))
}
return nil
}

View File

@@ -0,0 +1,24 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_omit_load_extension
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_OMIT_LOAD_EXTENSION
*/
import "C"
import (
"errors"
)
func (c *SQLiteConn) loadExtensions(extensions []string) error {
return errors.New("Extensions have been disabled for static builds")
}
func (c *SQLiteConn) LoadExtension(lib string, entry string) error {
return errors.New("Extensions have been disabled for static builds")
}

View File

@@ -0,0 +1,15 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_allow_uri_authority
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ALLOW_URI_AUTHORITY
#cgo LDFLAGS: -lm
*/
import "C"

View File

@@ -0,0 +1,16 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !windows
// +build sqlite_app_armor
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_API_ARMOR
#cgo LDFLAGS: -lm
*/
import "C"

View File

@@ -0,0 +1,21 @@
// +build sqlite_column_metadata
package sqlite3
/*
#ifndef USE_LIBSQLITE3
#cgo CFLAGS: -DSQLITE_ENABLE_COLUMN_METADATA
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
*/
import "C"
// ColumnTableName returns the table that is the origin of a particular result
// column in a SELECT statement.
//
// See https://www.sqlite.org/c3ref/column_database_name.html
func (s *SQLiteStmt) ColumnTableName(n int) string {
return C.GoString(C.sqlite3_column_table_name(s.s, C.int(n)))
}

View File

@@ -0,0 +1,15 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_foreign_keys
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_DEFAULT_FOREIGN_KEYS=1
#cgo LDFLAGS: -lm
*/
import "C"

14
vendor/github.com/mattn/go-sqlite3/sqlite3_opt_fts5.go generated vendored Normal file
View File

@@ -0,0 +1,14 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_fts5 fts5
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_FTS5
#cgo LDFLAGS: -lm
*/
import "C"

17
vendor/github.com/mattn/go-sqlite3/sqlite3_opt_icu.go generated vendored Normal file
View File

@@ -0,0 +1,17 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_icu icu
package sqlite3
/*
#cgo LDFLAGS: -licuuc -licui18n
#cgo CFLAGS: -DSQLITE_ENABLE_ICU
#cgo darwin CFLAGS: -I/usr/local/opt/icu4c/include
#cgo darwin LDFLAGS: -L/usr/local/opt/icu4c/lib
#cgo openbsd LDFLAGS: -lsqlite3
*/
import "C"

View File

@@ -0,0 +1,15 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_introspect
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_INTROSPECTION_PRAGMAS
#cgo LDFLAGS: -lm
*/
import "C"

View File

@@ -0,0 +1,13 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_json sqlite_json1 json1
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_JSON1
*/
import "C"

View File

@@ -0,0 +1,20 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build cgo
package sqlite3
// SQLitePreUpdateData represents all of the data available during a
// pre-update hook call.
type SQLitePreUpdateData struct {
Conn *SQLiteConn
Op int
DatabaseName string
TableName string
OldRowID int64
NewRowID int64
}

View File

@@ -0,0 +1,112 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_preupdate_hook
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_ENABLE_PREUPDATE_HOOK
#cgo LDFLAGS: -lm
#ifndef USE_LIBSQLITE3
#include <sqlite3-binding.h>
#else
#include <sqlite3.h>
#endif
#include <stdlib.h>
#include <string.h>
void preUpdateHookTrampoline(void*, sqlite3 *, int, char *, char *, sqlite3_int64, sqlite3_int64);
*/
import "C"
import (
"errors"
"unsafe"
)
// RegisterPreUpdateHook sets the pre-update hook for a connection.
//
// The callback is passed a SQLitePreUpdateData struct with the data for
// the update, as well as methods for fetching copies of impacted data.
//
// If there is an existing update hook for this connection, it will be
// removed. If callback is nil the existing hook (if any) will be removed
// without creating a new one.
func (c *SQLiteConn) RegisterPreUpdateHook(callback func(SQLitePreUpdateData)) {
if callback == nil {
C.sqlite3_preupdate_hook(c.db, nil, nil)
} else {
C.sqlite3_preupdate_hook(c.db, (*[0]byte)(unsafe.Pointer(C.preUpdateHookTrampoline)), unsafe.Pointer(newHandle(c, callback)))
}
}
// Depth returns the source path of the write, see sqlite3_preupdate_depth()
func (d *SQLitePreUpdateData) Depth() int {
return int(C.sqlite3_preupdate_depth(d.Conn.db))
}
// Count returns the number of columns in the row
func (d *SQLitePreUpdateData) Count() int {
return int(C.sqlite3_preupdate_count(d.Conn.db))
}
func (d *SQLitePreUpdateData) row(dest []interface{}, new bool) error {
for i := 0; i < d.Count() && i < len(dest); i++ {
var val *C.sqlite3_value
var src interface{}
// Initially I tried making this just a function pointer argument, but
// it's absurdly complicated to pass C function pointers.
if new {
C.sqlite3_preupdate_new(d.Conn.db, C.int(i), &val)
} else {
C.sqlite3_preupdate_old(d.Conn.db, C.int(i), &val)
}
switch C.sqlite3_value_type(val) {
case C.SQLITE_INTEGER:
src = int64(C.sqlite3_value_int64(val))
case C.SQLITE_FLOAT:
src = float64(C.sqlite3_value_double(val))
case C.SQLITE_BLOB:
len := C.sqlite3_value_bytes(val)
blobptr := C.sqlite3_value_blob(val)
src = C.GoBytes(blobptr, len)
case C.SQLITE_TEXT:
len := C.sqlite3_value_bytes(val)
cstrptr := unsafe.Pointer(C.sqlite3_value_text(val))
src = C.GoBytes(cstrptr, len)
case C.SQLITE_NULL:
src = nil
}
err := convertAssign(&dest[i], src)
if err != nil {
return err
}
}
return nil
}
// Old populates dest with the row data to be replaced. This works similar to
// database/sql's Rows.Scan()
func (d *SQLitePreUpdateData) Old(dest ...interface{}) error {
if d.Op == SQLITE_INSERT {
return errors.New("There is no old row for INSERT operations")
}
return d.row(dest, false)
}
// New populates dest with the replacement row data. This works similar to
// database/sql's Rows.Scan()
func (d *SQLitePreUpdateData) New(dest ...interface{}) error {
if d.Op == SQLITE_DELETE {
return errors.New("There is no new row for DELETE operations")
}
return d.row(dest, true)
}

View File

@@ -0,0 +1,21 @@
// Copyright (C) 2019 G.J.R. Timmer <gjr.timmer@gmail.com>.
// Copyright (C) 2018 segment.com <friends@segment.com>
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build !sqlite_preupdate_hook,cgo
package sqlite3
// RegisterPreUpdateHook sets the pre-update hook for a connection.
//
// The callback is passed a SQLitePreUpdateData struct with the data for
// the update, as well as methods for fetching copies of impacted data.
//
// If there is an existing update hook for this connection, it will be
// removed. If callback is nil the existing hook (if any) will be removed
// without creating a new one.
func (c *SQLiteConn) RegisterPreUpdateHook(callback func(SQLitePreUpdateData)) {
// NOOP
}

View File

@@ -0,0 +1,15 @@
// Copyright (C) 2019 Yasuhiro Matsumoto <mattn.jp@gmail.com>.
// Copyright (C) 2018 G.J.R. Timmer <gjr.timmer@gmail.com>.
//
// Use of this source code is governed by an MIT-style
// license that can be found in the LICENSE file.
// +build sqlite_secure_delete
package sqlite3
/*
#cgo CFLAGS: -DSQLITE_SECURE_DELETE=1
#cgo LDFLAGS: -lm
*/
import "C"

Some files were not shown because too many files have changed in this diff Show More