Commit Graph

397 Commits (6e77b062279afab8400da45b6fdc5b2787bdd801)

Author SHA1 Message Date
eXtremeSHOK 8eb05d1450
Fix for large Mailboxes to avoid timeouts 2
Missing ' (sorry my editor keeps correcting the '" to " )
2018-02-14 14:02:56 +02:00
eXtremeSHOK a37a8e3b2a
Fix for large Mailboxes to avoid timeouts
** I agree to the code of conduct and the contributory guidelines **

Tested with 80+GB single inbox via imap

``` --nofoldersizes --skipsize --fast ``` file and folder sizes are only used for statistics, which are completely useless. Before the actual data is transfered the message and folder sizes need to be calculated on a 200 000+ message inbox, this will almost never complete.

``` --buffersize 8192000 ``` sets the io buffer to 8mb, the default buffer is 4kbyte, this is speeds up syncs.

```  --skipheader 'X-*' ``` X headers vary wildly from system to system and do not ensure message uniqueness, they are not needed.

```  --split1 3000 --split2 3000 ```    split the requests in several parts on the server, 3000 is the number of messages handled per request.

``` --fastio1 --fastio2 ``` use fastio
2018-02-14 13:27:30 +02:00
Phoenix Eve Aspacio 42868b1710
Merge pull request #1037 from klausenbusk/empty
Remove "empty" folders
2018-02-14 08:15:13 +08:00
Kristian Klausen a0cdc1e4ff Remove "empty" folders
There seems to be no reason for this empty folders.
2018-02-13 23:45:49 +01:00
André Peters 66a3df16ed [Netfilter] Fixes empty f2b options 2018-02-09 10:32:12 +01:00
André Peters 83094a773e [Watchdog] Use socket to pipe to Rspamd 2018-02-08 22:57:29 +01:00
André Peters 004d262e41 [Rspamd] Create missing password include as placeholder 2018-02-08 22:57:17 +01:00
André Peters ac4982d706 [Dovecot] Use socket to pipe to Rspamd 2018-02-08 22:56:43 +01:00
André Peters a2ca550aa0 [Dovecot] Use socket instead of worker IP 2018-02-08 22:29:43 +01:00
André Peters e3854a8037 [Dockerapi] Fixes recent chang in exec_run return 2018-02-08 22:29:06 +01:00
root fae542534d Merge remote-tracking branch 'origin/master' into dev 2018-02-08 19:54:14 +01:00
Joshua Hesketh 2fc13bdfbe Fix imapsync for large mailboxes
Where there is a lot of folders in a mailbox the output may be larger
than TEXT will allow (64KiB). Instead use MEDIUMTEXT (16MiB) which
should be ample.

Additionally check that a sync isn't already running before starting
a new one.

Fixes: #1011
2018-02-08 15:24:04 +11:00
andre.peters cf1e46723a [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:37:10 +01:00
andre.peters 3f9e0a8023 [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:36:06 +01:00
andre.peters ae4ccd4d17 [Dovecot] Fixes CVE-2017-15132 - take 2 2018-02-01 23:35:55 +01:00
andre.peters 6ebcd00521 [Dovecot] Fixes CVE-2017-15132 2018-02-01 22:43:28 +01:00
andre.peters c5f9b065f6 [Dovecot] Fixes CVE-2017-15132 2018-02-01 22:27:48 +01:00
andre.peters a0d9efba00 [PHP-FPM] Remove old migration scripts 2018-02-01 13:40:59 +01:00
andre.peters f4ae354c0c [SOGo] Do not try to use foreign mailboxes as alias 2018-02-01 13:40:13 +01:00
andre.peters 38a819771b [Netfilter] Rename fail2ban to netfilter, use iptables-python 2018-02-01 13:39:27 +01:00
andre.peters 0773448b35 [Dockerapi] Fix jsonify output 2018-02-01 13:38:42 +01:00
André Peters 912ba9b4ff
Fixes #979 2018-01-29 14:42:51 +01:00
Michael Kuron c30448c4d8 Merge branch 'master' of https://github.com/andryyy/mailcow-dockerized into recipient_map
Conflicts:
	data/web/inc/init_db.inc.php
2018-01-27 17:22:08 +01:00
andre.peters 7433b6dc91 [ClamAV] Build from source, fix bugs...
[Compose] New clamd-mailcow image
2018-01-27 10:26:12 +01:00
andre.peters 04f9d74339 [Web, Dovecot] Add new options to imapsync, other minor changes to forms, partly fixes #955 2018-01-24 12:59:11 +01:00
andre.peters 1aaa5682b4 [Fail2ban] Allow to set subnet size for banned networks 2018-01-24 09:11:33 +01:00
andre.peters f2f4dabce4 [Postfix] postconf wrapper for correct config location, fixes #949 2018-01-24 09:10:43 +01:00
andre.peters 46aafff627 [ClamAV] Outsource config 2018-01-24 08:40:13 +01:00
Michael Kuron e86565e283 Expose Postfix's recipient_canonical_maps through web UI 2018-01-23 20:02:31 +01:00
andre.peters a7a7b3f3fd [Postfix] Use name instead of IP 2018-01-21 15:01:51 +01:00
andre.peters d24bb16947 [Watchdog] Check PHP-FPM port 9000 and 9001 2018-01-21 15:01:35 +01:00
andre.peters 737c41379f [PHP-FPM] Move opcache config to local config file, define new PECL versions 2018-01-21 15:01:02 +01:00
andre.peters 83a21259f7 [Rspamd] Use names instead of IPs 2018-01-21 15:00:05 +01:00
andre.peters 08c8976a95 [SOGo] Show shared aliases and "allow to send as" addresses as FROM fields in SOGo 2018-01-21 14:58:16 +01:00
andre.peters 8419266678 [Web] Important fixes for quarantaine; other minor changes 2018-01-17 15:23:33 +01:00
andre.peters 09b6c20bad [DockerAPI] Hide stderr from rspamadm output 2018-01-14 18:44:06 +01:00
andre.peters 5d5d36fc60 [Dovecot] Revert to 2.2 to fix various errors 2018-01-14 10:44:06 +01:00
andre.peters 38aaeeb30b [Dovecot] Dovecot 2.3 from git for temp fixes
[ACME] Use -a switch (thanks to jas8522!)
2018-01-14 10:34:06 +01:00
andre.peters 42923698fd [Dovcot] Fixes for v2.3 2018-01-13 22:49:50 +01:00
andre.peters ceef6d6fd3 [Dovecot] Dovecot 2.3, Pigeonhole 0.5
[Watchdog] Fix Dovecot health check
2018-01-09 11:15:52 +01:00
andre.peters b7a23a28fd Merge branch 'dev' of https://github.com/mailcow/mailcow-dockerized into dev 2018-01-08 22:01:39 +01:00
andre.peters 1e9cae9084 [Compose] New images with LOG_LINES
[Update] Do not add empty line in each loop
2018-01-08 22:00:54 +01:00
Michael Kuron cde250a989 Properly wait for MySQL to come up before starting SOGo 2018-01-08 19:56:44 +01:00
andre.peters db032af698 [ACME] Fix script 2017-12-31 17:17:46 +01:00
andre.peters 8d56534e76 [Postfix] Don't try to authenticate to relayhosts without username, fixes #725 2017-12-25 10:18:49 +01:00
Oratorian 236e4d4a36
See Issue #826
Fixes dockerapi-mailcow_1 | raise TypeError('port must be an integer')

Containers are now restarting.
2017-12-18 16:41:04 +01:00
andre.peters 52f4f850cf [Postfix] Fix missing authentication data for relayhosts when sender domain is alias domain 2017-12-17 17:45:12 +01:00
andre.peters a771d66889 [Dovecot] Update imapsync 2017-12-17 17:45:05 +01:00
andre.peters 2994b94b6c [Docker API] Remove logs, remove env var for compose project name 2017-12-11 09:43:01 +01:00
andre.peters 6c67b9df82 Replace name by IP, remove unused tables 2017-12-09 22:30:18 +01:00
andre.peters 2519738094 Various changes... 2017-12-09 13:15:24 +01:00
Peter Schiffer 7d6fc8e6b6 [Dockerapi] Auto detect version of docker server
Some older versions of docker need specific version of client api to be able to
communicate. This change allows automatically detect and set version of API to
match server version of API.

Fixes #765
2017-12-01 23:41:37 +01:00
André ade4b9e7ae [Postfix, Web] Feature: BCC maps 2017-11-19 15:13:43 +01:00
André 5211ab10ed [ACME] Fix license issue 2017-11-16 14:57:17 +01:00
André c45ea5433f [Rspamd] Base on xenial to fix mime types problems 2017-11-15 12:29:18 +01:00
André fe845ee56d [Fail2ban] Fix fail2ban container 2017-11-14 19:50:20 +01:00
André 84a7a1a2e7 [Compose] New images, Nginx checks for SOGo before bootstrapping
[PHP-FPM] Some more modules (primarily for Horde)
[Fail2ban] Do not log matches of local and private ips
[Watchdog] Some changes in log system for further processing (wip)
[ACME] Fixes #745
2017-11-14 10:44:22 +01:00
André 60e97503f7 [Web, Dovecot] Show wether a sync job is running, validate min max input attr and validate these values 2017-11-08 11:07:32 +01:00
André a36a8828c2 [Dovecot] Specify supervisord user 2017-11-05 12:19:18 +01:00
André 1ef10f1358 [PHP-FPM] Include net_sieve, test removal of usr/src/php for size 2017-11-03 20:27:43 +01:00
André a9f64a3472 [Dockerapi] Return answers in json 2017-11-03 20:26:09 +01:00
André b32e5adcc5 [Dovecot] sieve_before/after maps in sql, changed dict names 2017-11-03 20:25:38 +01:00
André 1e9bc49f2c [Rspamd] Echo dummy for fowardingshosts map; Use higher map reading interval;
[Dockerapi] Exit on sigterm;
[Watchdog] Wait for dockerapi-mailcow to be online
2017-10-27 11:22:39 +02:00
André 04cb033f0a [PHP-FPM] Add imagemagic 2017-10-21 10:10:27 +02:00
André a110e2ea0f [ACME] Fix detection of orphaned SANs and add tini 2017-10-21 10:08:20 +02:00
André 6b6470fe54 [Rspamd] Use tini 2017-10-15 09:31:19 +02:00
André ac413058c1 [Watchdog] Fix kill -0 check, use tini and send kill request to tini instead of parent pid, sleep longer 2017-10-14 23:26:08 +02:00
André c5dd30b058 [ClamAV] Use tini, check if background procs are running, use pipe to output to stdout 2017-10-14 23:25:29 +02:00
andryyy c5054ae7ed [Watchdog] Ignore null name in jq
[Nginx] Merge sites
[Scripts] Nextcloud helper script (testing!)
2017-10-11 22:56:22 +02:00
andryyy 7c46d6548b [Dovecot] Ignore watchdog IP in logs, filter by syslog-ng 2017-10-11 11:23:20 +02:00
andryyy e107cbef5e [Postfix] Fix sending as alias, when alias is in alias domains, cleanup 2017-10-11 11:22:52 +02:00
andryyy 2862b43c81 [Watchdog] Fix watchdog to fit non-exposed PHP 2017-10-09 15:54:54 +02:00
andryyy 6110ac386f [SOGo] Use official nightly; [PHP-FPM] Fix expose=off 2017-10-09 15:45:48 +02:00
andryyy 72995ff98e [PHP-FPM] Include more modules for upcoming features and Nextcloud support, drop ro flag; [Watchdog] Some fixes and changes 2017-10-08 22:47:52 +02:00
andryyy ef9953898c [ACME, Watchdog, DockerAPI] Use only limited Docker API 2017-10-06 13:32:49 +02:00
andryyy 3ae0b16845 [Web, DockerAPI] Be more like official Docker API 2017-10-06 10:20:40 +02:00
andryyy fc18d153cd [Compose, DockerAPI, Web, Watchdog] Watchdog may send notification mails (todo: docs), DockerAPI via Flesk for limited access 2017-10-05 23:38:33 +02:00
andryyy 9860d44d04 [Watchdog] Do also log errors to Redis if availble 2017-10-04 23:18:51 +02:00
andryyy 2dc8306b69 [Postfix] Remove old socket 2017-10-04 23:15:26 +02:00
andryyy 82ac5fa063 [SOGo] Remove supervisord API 2017-10-04 13:04:35 +02:00
andryyy 9b4ed6b21c [PHP-FPM] Include Docker api for better SOGo status handling and future changes 2017-10-04 13:04:15 +02:00
andryyy da987e5b48 [Postfix] Forgot 'not' in filter 2017-10-03 16:54:18 +02:00
andryyy c59d03fcb3 [Watchdog] Skip when use_watchdog=n 2017-10-03 12:07:48 +02:00
andryyy 68d7fa1504 [Watchdog] Skip when use_watchdog=n 2017-10-03 12:05:38 +02:00
Michael Kuron 752a571607 Merge pull request #638 from mkuron/acme
ACME needs to wait for MySQL to be ready
2017-09-28 12:58:07 +02:00
Michael Kuron ae79445ec0 ACME needs to wait for MySQL to be ready 2017-09-27 19:48:25 +02:00
andryyy 337c9e350e [Watchdog] Reset diff, new image 2017-09-22 16:40:02 +02:00
andryyy 62524150d2 [ACME] Add timestamps, check if acme account key is valid
[Postfix] Ignore local0
[Watchdog] Add Rspamd checks
2017-09-21 19:30:09 +02:00
andryyy ab850dc901 [ACME] Detect and fix invalid registration 2017-09-21 09:46:09 +02:00
andryyy 41d2a16571 [Watchdog] Script was not executable 2017-09-20 23:36:04 +02:00
andryyy f511cb0f63 [Watchdog] More fixes and or changes 2017-09-20 23:24:56 +02:00
andryyy fd1955edca [Fail2ban] Add variable name 2017-09-20 23:24:39 +02:00
andryyy b6e84fac3a Sleep instead of stopping containers to prevent restarts 2017-09-20 12:50:50 +02:00
andryyy df5c79c3f1 Fixes for watchdog! 2017-09-20 12:27:24 +02:00
andryyy e70d5b9206 Fix watchdog 2017-09-20 11:05:23 +02:00
andryyy a8fb1d3f4f Add experimental watchdog 2017-09-20 10:56:49 +02:00
andryyy d0c0cd4992 [Rspamd] Fix user settings... 2017-09-16 23:27:13 +02:00
andryyy 1b974bc8d1 [Compose] New images 2017-09-16 23:05:33 +02:00
andryyy 2b97305f6d [ACME] Sleep, don't exit 2017-09-16 13:17:48 +02:00
andryyy 762f18e913 [Clamd] Use Dockerds own init system 2017-09-14 23:13:24 +02:00