Commit Graph

447 Commits (668a0927acdc9f3e6ad63c524a388fc7b0b678c0)

Author SHA1 Message Date
eXtremeSHOK 8eb05d1450
Fix for large Mailboxes to avoid timeouts 2
Missing ' (sorry my editor keeps correcting the '" to " )
2018-02-14 14:02:56 +02:00
eXtremeSHOK a37a8e3b2a
Fix for large Mailboxes to avoid timeouts
** I agree to the code of conduct and the contributory guidelines **

Tested with 80+GB single inbox via imap

``` --nofoldersizes --skipsize --fast ``` file and folder sizes are only used for statistics, which are completely useless. Before the actual data is transfered the message and folder sizes need to be calculated on a 200 000+ message inbox, this will almost never complete.

``` --buffersize 8192000 ``` sets the io buffer to 8mb, the default buffer is 4kbyte, this is speeds up syncs.

```  --skipheader 'X-*' ``` X headers vary wildly from system to system and do not ensure message uniqueness, they are not needed.

```  --split1 3000 --split2 3000 ```    split the requests in several parts on the server, 3000 is the number of messages handled per request.

``` --fastio1 --fastio2 ``` use fastio
2018-02-14 13:27:30 +02:00
Phoenix Eve Aspacio 42868b1710
Merge pull request #1037 from klausenbusk/empty
Remove "empty" folders
2018-02-14 08:15:13 +08:00
Kristian Klausen a0cdc1e4ff Remove "empty" folders
There seems to be no reason for this empty folders.
2018-02-13 23:45:49 +01:00
André Peters 66a3df16ed [Netfilter] Fixes empty f2b options 2018-02-09 10:32:12 +01:00
André Peters 83094a773e [Watchdog] Use socket to pipe to Rspamd 2018-02-08 22:57:29 +01:00
André Peters 004d262e41 [Rspamd] Create missing password include as placeholder 2018-02-08 22:57:17 +01:00
André Peters ac4982d706 [Dovecot] Use socket to pipe to Rspamd 2018-02-08 22:56:43 +01:00
André Peters a2ca550aa0 [Dovecot] Use socket instead of worker IP 2018-02-08 22:29:43 +01:00
André Peters e3854a8037 [Dockerapi] Fixes recent chang in exec_run return 2018-02-08 22:29:06 +01:00
root fae542534d Merge remote-tracking branch 'origin/master' into dev 2018-02-08 19:54:14 +01:00
Joshua Hesketh 2fc13bdfbe Fix imapsync for large mailboxes
Where there is a lot of folders in a mailbox the output may be larger
than TEXT will allow (64KiB). Instead use MEDIUMTEXT (16MiB) which
should be ample.

Additionally check that a sync isn't already running before starting
a new one.

Fixes: #1011
2018-02-08 15:24:04 +11:00
andre.peters cf1e46723a [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:37:10 +01:00
andre.peters 3f9e0a8023 [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:36:06 +01:00
andre.peters ae4ccd4d17 [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:35:55 +01:00
andre.peters 6ebcd00521 [Dovecot] Fixes CVE-2017-15132 2018-02-01 22:43:28 +01:00
andre.peters c5f9b065f6 [Dovecot] Fixes CVE-2017-15132 2018-02-01 22:27:48 +01:00
andre.peters a0d9efba00 [PHP-FPM] Remove old migration scripts 2018-02-01 13:40:59 +01:00
andre.peters f4ae354c0c [SOGo] Do not try to use foreign mailboxes as alias 2018-02-01 13:40:13 +01:00
andre.peters 38a819771b [Netfilter] Rename fail2ban to netfilter, use iptables-python 2018-02-01 13:39:27 +01:00
andre.peters 0773448b35 [Dockerapi] Fix jsonify output 2018-02-01 13:38:42 +01:00
André Peters 912ba9b4ff
Fixes #979 2018-01-29 14:42:51 +01:00
Michael Kuron c30448c4d8 Merge branch 'master' of https://github.com/andryyy/mailcow-dockerized into recipient_map
Conflicts:
	data/web/inc/init_db.inc.php
2018-01-27 17:22:08 +01:00
andre.peters 7433b6dc91 [ClamAV] Build from source, fix bugs...
[Compose] New clamd-mailcow image
2018-01-27 10:26:12 +01:00
andre.peters 04f9d74339 [Web, Dovecot] Add new options to imapsync, other minor changes to forms, partly fixes #955 2018-01-24 12:59:11 +01:00
andre.peters 1aaa5682b4 [Fail2ban] Allow to set subnet size for banned networks 2018-01-24 09:11:33 +01:00
andre.peters f2f4dabce4 [Postfix] postconf wrapper for correct config location, fixes #949 2018-01-24 09:10:43 +01:00
andre.peters 46aafff627 [ClamAV] Outsource config 2018-01-24 08:40:13 +01:00
Michael Kuron e86565e283 Expose Postfix's recipient_canonical_maps through web UI 2018-01-23 20:02:31 +01:00
andre.peters a7a7b3f3fd [Postfix] Use name instead of IP 2018-01-21 15:01:51 +01:00
andre.peters d24bb16947 [Watchdog] Check PHP-FPM port 9000 and 9001 2018-01-21 15:01:35 +01:00
andre.peters 737c41379f [PHP-FPM] Move opcache config to local config file, define new PECL versions 2018-01-21 15:01:02 +01:00
andre.peters 83a21259f7 [Rspamd] Use names instead of IPs 2018-01-21 15:00:05 +01:00
andre.peters 08c8976a95 [SOGo] Show shared aliases and "allow to send as" addresses as FROM fields in SOGo 2018-01-21 14:58:16 +01:00
andre.peters 8419266678 [Web] Important fixes for quarantaine; other minor changes 2018-01-17 15:23:33 +01:00
andre.peters 09b6c20bad [DockerAPI] Hide stderr from rspamadm output 2018-01-14 18:44:06 +01:00
andre.peters 5d5d36fc60 [Dovecot] Revert to 2.2 to fix various errors 2018-01-14 10:44:06 +01:00
andre.peters 38aaeeb30b [Dovecot] Dovecot 2.3 from git for temp fixes
[ACME] Use -a switch (thanks to jas8522!)
2018-01-14 10:34:06 +01:00
andre.peters 42923698fd [Dovcot] Fixes for v2.3 2018-01-13 22:49:50 +01:00
andre.peters ceef6d6fd3 [Dovecot] Dovecot 2.3, Pigeonhole 0.5
[Watchdog] Fix Dovecot health check
2018-01-09 11:15:52 +01:00
andre.peters b7a23a28fd Merge branch 'dev' of https://github.com/mailcow/mailcow-dockerized into dev 2018-01-08 22:01:39 +01:00
andre.peters 1e9cae9084 [Compose] New images with LOG_LINES
[Update] Do not add empty line in each loop
2018-01-08 22:00:54 +01:00
Michael Kuron cde250a989 Properly wait for MySQL to come up before starting SOGo 2018-01-08 19:56:44 +01:00
andre.peters db032af698 [ACME] Fix script 2017-12-31 17:17:46 +01:00
andre.peters 8d56534e76 [Postfix] Don't try to authenticate to relayhosts without username, fixes #725 2017-12-25 10:18:49 +01:00
Oratorian 236e4d4a36
See Issue #826
Fixes dockerapi-mailcow_1 | raise TypeError('port must be an integer')

Containers are now restarting.
2017-12-18 16:41:04 +01:00
andre.peters 52f4f850cf [Postfix] Fix missing authentication data for relayhosts when sender domain is alias domain 2017-12-17 17:45:12 +01:00
andre.peters a771d66889 [Dovecot] Update imapsync 2017-12-17 17:45:05 +01:00
andre.peters 2994b94b6c [Docker API] Remove logs, remove env var for compose project name 2017-12-11 09:43:01 +01:00
andre.peters 6c67b9df82 Replace name by IP, remove unused tables 2017-12-09 22:30:18 +01:00
andre.peters 2519738094 Various changes... 2017-12-09 13:15:24 +01:00
Peter Schiffer 7d6fc8e6b6 [Dockerapi] Auto detect version of docker server
Some older versions of docker need specific version of client api to be able to
communicate. This change allows automatically detect and set version of API to
match server version of API.

Fixes #765
2017-12-01 23:41:37 +01:00
André ade4b9e7ae [Postfix, Web] Feature: BCC maps 2017-11-19 15:13:43 +01:00
André 5211ab10ed [ACME] Fix license issue 2017-11-16 14:57:17 +01:00
André c45ea5433f [Rspamd] Base on xenial to fix mime types problems 2017-11-15 12:29:18 +01:00
André fe845ee56d [Fail2ban] Fix fail2ban container 2017-11-14 19:50:20 +01:00
André 84a7a1a2e7 [Compose] New images, Nginx checks for SOGo before bootstrapping
[PHP-FPM] Some more modules (primarily for Horde)
[Fail2ban] Do not log matches of local and private ips
[Watchdog] Some changes in log system for further processing (wip)
[ACME] Fixes #745
2017-11-14 10:44:22 +01:00
André 60e97503f7 [Web, Dovecot] Show wether a sync job is running, validate min max input attr and validate these values 2017-11-08 11:07:32 +01:00
André a36a8828c2 [Dovecot] Specify supervisord user 2017-11-05 12:19:18 +01:00
André 1ef10f1358 [PHP-FPM] Include net_sieve, test removal of usr/src/php for size 2017-11-03 20:27:43 +01:00
André a9f64a3472 [Dockerapi] Return answers in json 2017-11-03 20:26:09 +01:00
André b32e5adcc5 [Dovecot] sieve_before/after maps in sql, changed dict names 2017-11-03 20:25:38 +01:00
André 1e9bc49f2c [Rspamd] Echo dummy for fowardingshosts map; Use higher map reading interval;
[Dockerapi] Exit on sigterm;
[Watchdog] Wait for dockerapi-mailcow to be online
2017-10-27 11:22:39 +02:00
André 04cb033f0a [PHP-FPM] Add imagemagic 2017-10-21 10:10:27 +02:00
André a110e2ea0f [ACME] Fix detection of orphaned SANs and add tini 2017-10-21 10:08:20 +02:00
André 6b6470fe54 [Rspamd] Use tini 2017-10-15 09:31:19 +02:00
André ac413058c1 [Watchdog] Fix kill -0 check, use tini and send kill request to tini instead of parent pid, sleep longer 2017-10-14 23:26:08 +02:00
André c5dd30b058 [ClamAV] Use tini, check if background procs are running, use pipe to output to stdout 2017-10-14 23:25:29 +02:00
andryyy c5054ae7ed [Watchdog] Ignore null name in jq
[Nginx] Merge sites
[Scripts] Nextcloud helper script (testing!)
2017-10-11 22:56:22 +02:00
andryyy 7c46d6548b [Dovecot] Ignore watchdog IP in logs, filter by syslog-ng 2017-10-11 11:23:20 +02:00
andryyy e107cbef5e [Postfix] Fix sending as alias, when alias is in alias domains, cleanup 2017-10-11 11:22:52 +02:00
andryyy 2862b43c81 [Watchdog] Fix watchdog to fit non-exposed PHP 2017-10-09 15:54:54 +02:00
andryyy 6110ac386f [SOGo] Use official nightly; [PHP-FPM] Fix expose=off 2017-10-09 15:45:48 +02:00
andryyy 72995ff98e [PHP-FPM] Include more modules for upcoming features and Nextcloud support, drop ro flag; [Watchdog] Some fixes and changes 2017-10-08 22:47:52 +02:00
andryyy ef9953898c [ACME, Watchdog, DockerAPI] Use only limited Docker API 2017-10-06 13:32:49 +02:00
andryyy 3ae0b16845 [Web, DockerAPI] Be more like official Docker API 2017-10-06 10:20:40 +02:00
andryyy fc18d153cd [Compose, DockerAPI, Web, Watchdog] Watchdog may send notification mails (todo: docs), DockerAPI via Flesk for limited access 2017-10-05 23:38:33 +02:00
andryyy 9860d44d04 [Watchdog] Do also log errors to Redis if availble 2017-10-04 23:18:51 +02:00
andryyy 2dc8306b69 [Postfix] Remove old socket 2017-10-04 23:15:26 +02:00
andryyy 82ac5fa063 [SOGo] Remove supervisord API 2017-10-04 13:04:35 +02:00
andryyy 9b4ed6b21c [PHP-FPM] Include Docker api for better SOGo status handling and future changes 2017-10-04 13:04:15 +02:00
andryyy da987e5b48 [Postfix] Forgot 'not' in filter 2017-10-03 16:54:18 +02:00
andryyy c59d03fcb3 [Watchdog] Skip when use_watchdog=n 2017-10-03 12:07:48 +02:00
andryyy 68d7fa1504 [Watchdog] Skip when use_watchdog=n 2017-10-03 12:05:38 +02:00
Michael Kuron 752a571607 Merge pull request #638 from mkuron/acme
ACME needs to wait for MySQL to be ready
2017-09-28 12:58:07 +02:00
Michael Kuron ae79445ec0 ACME needs to wait for MySQL to be ready 2017-09-27 19:48:25 +02:00
andryyy 337c9e350e [Watchdog] Reset diff, new image 2017-09-22 16:40:02 +02:00
andryyy 62524150d2 [ACME] Add timestamps, check if acme account key is valid
[Postfix] Ignore local0
[Watchdog] Add Rspamd checks
2017-09-21 19:30:09 +02:00
andryyy ab850dc901 [ACME] Detect and fix invalid registration 2017-09-21 09:46:09 +02:00
andryyy 41d2a16571 [Watchdog] Script was not executable 2017-09-20 23:36:04 +02:00
andryyy f511cb0f63 [Watchdog] More fixes and or changes 2017-09-20 23:24:56 +02:00
andryyy fd1955edca [Fail2ban] Add variable name 2017-09-20 23:24:39 +02:00
andryyy b6e84fac3a Sleep instead of stopping containers to prevent restarts 2017-09-20 12:50:50 +02:00
andryyy df5c79c3f1 Fixes for watchdog! 2017-09-20 12:27:24 +02:00
andryyy e70d5b9206 Fix watchdog 2017-09-20 11:05:23 +02:00
andryyy a8fb1d3f4f Add experimental watchdog 2017-09-20 10:56:49 +02:00
andryyy d0c0cd4992 [Rspamd] Fix user settings... 2017-09-16 23:27:13 +02:00
andryyy 1b974bc8d1 [Compose] New images 2017-09-16 23:05:33 +02:00
andryyy 2b97305f6d [ACME] Sleep, don't exit 2017-09-16 13:17:48 +02:00
andryyy 762f18e913 [Clamd] Use Dockerds own init system 2017-09-14 23:13:24 +02:00
andryyy eeaa48a729 [PHP-FPM] Use valid user for mysqladmin ping
[SOGo] Use valid user for mysqladmin ping
2017-09-12 20:57:18 +02:00
andryyy 1ffed58956 [ACME] New version, better IPv4 detection 2017-09-11 21:51:17 +02:00
andryyy ed4e8d301c Fix tabs 2017-09-08 18:41:02 +02:00
andryyy b8c4093702 [Web] Allow a spam score up to 2000, fixes #556 2017-09-07 20:20:36 +02:00
Tobias fcd0efc265 Change primary name of SSL certificate
Hostname as primary name for the SSL certificate, all other names will
be added as SAN
2017-09-03 19:41:47 +02:00
andryyy 1f90433429 [Dovecot, Postfix, SOGo] Fix redis log cleanup, fixes #542 2017-09-03 11:57:58 +02:00
andryyy 446907944a [Postfix] Fix missing ltrim 2017-09-01 23:40:23 +02:00
andryyy 008d500f4d [Compose, Dovecot] Fixed missing ltrim 2017-09-01 23:39:51 +02:00
andryyy 7351fcf1e3 [Compose, SOGo] New version, ltrim logs 2017-09-01 23:33:12 +02:00
andryyy 4d7bb26874 [Postfix] Add null rcpt for watchdog 2017-09-01 12:22:29 +02:00
andryyy 08d6b97ffa [Dovecot] New Dovecot and Pigeonhole versions; include delete2 option in imapsync 2017-08-31 10:36:10 +02:00
andryyy 4a0e3a433b [Fail2ban] Remove rule to detect disconnects without authentication 2017-08-30 22:27:33 +02:00
andryyy b1213c51d7 [Rspamd] Dynamic ratelimit fixed, removed async redis request; Ready to implement per-user ratelimits via UI (tbd) 2017-08-30 21:42:39 +02:00
andryyy 7a2c0f2ee7 [Acme] Skip backup mx domains 2017-08-18 09:57:25 +02:00
André Peters 92eefc1288 Merge pull request #486 from mkuron/patch-1
Disable ClamAV phishing filter
2017-08-02 15:54:16 +02:00
Michael Kuron dec64eef32 Disable ClamAV phishing filter
ClamAV has a phishing filter built in, but it generates too many false positives. We can disable it without risk because rspamd also checks for phishing itself.
2017-08-02 14:42:53 +02:00
andryyy 2eed7c05a1 [Dovecot] Push version, fix cronjob (fixes #310) 2017-08-01 14:02:00 +02:00
andryyy c7484434dd [Dovecot] Remove master.pid if not running; Check for active imapsync besides lock file; Reconnect MySQL connection in imapsync_cron.pl 2017-07-31 08:19:02 +02:00
andryyy 33bf9f5c5d [Rspamd] Push image version 1.5, Rspamd 1.6.3 2017-07-26 23:07:01 +02:00
andryyy 8da02378b1 [Rspamd] Use Nginx IP in settings map 2017-07-26 23:04:49 +02:00
andryyy 03c614f749 [Postfix] Use Nginx IP in whitelist_forwardinghosts.sh 2017-07-26 23:04:27 +02:00
andryyy b629089ff8 [SOGo] Rename reconf-domains to bootstrap-sogo and exec sogod after a successful bootstrap; Wait for SOGo to die and free listener 2017-07-24 23:25:04 +02:00
andryyy 5d5646df0c [Compose, SOGo] Update SOGo, give SOGo more time for a graceful restart, autorestart on any exit code 2017-07-24 22:53:23 +02:00
andryyy 83d485dd94 [Web, Postfix, Compose] Allow to add relayhosts per domain (+ plain and login authentication) 2017-07-22 20:39:54 +02:00
andryyy ed33cb5f57 [Rspamd] ARC: Disallow login/domain mismatch 2017-07-21 11:03:35 +02:00
andryyy 5f5872f78b [Rspamd] Initial custom ratelimit support 2017-07-13 12:54:53 +02:00
andryyy e6727b1fd6 [ACME] Iterate alias domains, use hostname in subject field 2017-07-13 12:51:52 +02:00
andryyy 9e92c4a2ad [Dovecot] Do not keep persistent logs in a container 2017-07-11 17:09:31 +02:00
andryyy f8ae5158cb [Postfix] Do not keep persistent logs in a container 2017-07-11 17:09:20 +02:00
andryyy a31819fd6c [SOGo] Log to a pipe to not keep logs in a container 2017-07-11 17:08:06 +02:00
andryyy 51660589d4 [Rspamd] Push version 1.3 2017-07-09 22:28:36 +02:00
andryyy a324b1a385 [Rspamd] Fix DKIM siging (base64 hash line folding) 2017-07-06 15:57:33 +02:00
andryyy 69da02c144 [Dovecot] Important fix for Pigeonhole (downgrade) 2017-07-05 19:13:07 +02:00
andryyy 7f47af1d60 Fix destination hash 2017-07-05 12:31:52 +02:00
andryyy 87cc5f54ff [Postfix] Fix redis log destination hash 2017-07-05 12:03:12 +02:00
andryyy 2fadfee61a [SOGo] Remove thunderbird plugin generation, will move to docs 2017-07-05 10:22:48 +02:00
andryyy de14d30e6b [ACME] Fix exit command (was using echo), fix for duplicate SAN (filter) 2017-07-04 21:32:58 +02:00
andryyy dc463c3dda [Multiple] Push multiple logs to Redis channel for fail2ban-mailcow to read. Enables Fail2ban independently of used Docker logging driver. 2017-07-04 18:08:20 +02:00
andryyy ed11e7586e [Clamd] Add SKIP_CLAMD variable to disable Clamd start 2017-07-04 18:05:44 +02:00
andryyy ace247b3b9 [ACME] Unset name arrays before reusing them 2017-07-03 10:20:09 +02:00
andryyy 2cf9f71613 [ACME] Revert fix for empty additional_san, fix skip_ip_check in ACME 2017-07-02 20:18:22 +02:00
andryyy cf902854d7 Merge, conflict fixed 2017-07-02 11:22:35 +02:00
andryyy b7cb4ac9d5 [Fail2ban] Added more regex to match failed or disallowed logins to Dovecot, changed Mailcow to mailcow 2017-07-02 11:10:35 +02:00
andryyy ae5ce6568d [Dovecot] Dovecot 2.2.31, Pigeonhole 0.4.19 2017-06-30 20:30:30 +02:00
andryyy 5ab11c0c1e [ACME] Show err instead of empty var when IP lookup failed 2017-06-30 20:29:55 +02:00
andryyy d2048ccf20 [ACME] Be more verbose about IP address 2017-06-29 21:22:01 +02:00
andryyy a6b60aebb8 [Fai2ban] Added auto-detection for container names; Allow multiple rules for each container; log rule id and container on match 2017-06-29 11:30:14 +02:00
andryyy 9040d456ed [acme-mailcow] Auto-detect container ids for restart; Restart containers after restore 2017-06-29 10:25:32 +02:00
andryyy 3d652dd3d0 Added more checks for acme-mailcow 2017-06-29 00:56:51 +02:00
andryyy 6d8438c01c - More checks for acme-mailcow (verify hashes)
- Autodiscover configuration file: Merge array from vars.local.inc.php
- Push acme-mailcow to 1.6
2017-06-28 23:22:51 +02:00