The overarching question
How do I see what is 'bad' about a 400 - bad request
?
Info about the error
- When I click around the WordPress-backend, then between 3 and 7 requests (out of 95-100) give me a:
400 - Bad request
-error, upon every page load. But it varies which request that fails, from refresh to refresh?!
- I've been battling this issue for 8 hours (ish). I feel like I've tried everything.
- It happens both in Chrome and Firefox
- It doesn't happen on another computer with an identical setup.
- It's mainly static files, where the requests fail (but not exclusively).
- There are 10-15 other developers also working on it, they haven't seen it either (same branch, almost identical database, same Vagrant setup).
System info
- WordPress-installation
- Apache-server
- Running locally on a Vagrant box (Ubuntu VirtualBox)
Solution attempt 1: See tendencies between which files it fails to load
I can't see any tendencies there. Here are 4 random refreshes.
Refresh1
api-client-1740.js
backbone.min.js
styleGuide-1740.js
escape-html.js
admin-script.min.js
quicktags.js
admin-bar.js
hooks.js
debug-bar-js.dev.js
font-awesome.css
nav-menus.css
list-tables.css
Refresh2
nav-menus.css
about.css
admin-menu.css
Refresh3
tags-suggest.js
i18n.js
wp-polyfill.js
themes.css
list-tables.css
Refresh4
"Image loadingAnimation.gif"
flame.png
api-client-1740.js
element.min.js
admin-script.min.js
moment.min.js
mouse.min.js
lodash.min.js
underscore.min.js
hoverintent-js.min.js
adminbar-1740.css
admin-global-1740.css
And sometimes none of the requests fails.
Solution attempt 2: Inspect the request in the network-tab
I can find a reason for the 400 Bad request
in there. Here is an example of a request that returned 400 Bad request
(where I've redacted sensitive info).
Request
GET /wp/wp-admin/load-styles.php?c=0&dir=ltr&load%5Bchunk_0%5D=dashicons,admin-bar,site-health,common,forms,admin-menu,dashboard,list-tables,edit,revisions,media,themes,about,nav-menus,wp-poi&load%5Bchunk_1%5D=nter,widgets,site-icon,l10n,buttons,wp-auth-check&ver=5.8.3 HTTP/1.1
Host: mydomain.test
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0
Accept: text/css,*/*;q=0.1
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://mydomain.test/wp/wp-admin/
DNT: 1
Connection: keep-alive
Cookie: wordpress_a8d2d12312312379974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cc70240d7f46d0dc5b6112554da95f69cee61180ebb2b3a93326cb42d02937f3f; wordpress_test_cookie=WP%20Cookie%20check; wordpress_logged_in_a8d2dfda29fb0e79974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cd16311d6028a3550a8777754a99a1e2d09bffb05f45f36854038387bc0fdce43; wp-settings-2=posts_list_mode%3Dlist%26ampampampamplibraryContent%3Dbrowse%26ampampampampeditor%3Dtinymce%26ampampampampmfold%3Do%26libraryContent%3Dbrowse; wp-settings-time-2=1652706157; CookieConsent={stamp:%27bs7eydGl1MNS0O2BJZiPnLkz4NSuHl+9jWJceT1L8gnBXxa3B+MlsA==%27%2Cnecessary:true%2Cpreferences:true%2Cstatistics:true%2Cmarketing:true%2Cver:1%2Cutc:1652701927276%2Ciab2:%27CPZD8gAPZD8gACGABBENCPCsAP_AAH_AAAAAIwtd_X__bX9j-_5_bft0eY1P9_r3_-QzjhfNs-8F3L_W_L0Xw2E7NF36pq4KuR4Eu3LBIQNlHMHUTUmwaokVrzHsak2cpyNKJ7LEknMZO2dYGH9Pn9lDuYKY7_5___bx3D-v_t_-39T378Xf3_d5_2_--vCfV599jbn9fV_7_9nP___9v-_8_________gjAASYal5AF2JY4MmkaRQogRhWEhVAoAKKAYWiKwAcHBTsrAJdQQsAEAqAjAiBBiCjBgEAAgEASERASAFggEQBEAgABAAiAQgAImAQWAFgYBAAKAaFiAFAAIEhBkQERymBARIlFBLZWIJQV7GmEAdZYAUCiMioAESAAAkBASFg5jgCQEuFkgSYoXyAEYIUAAAAA.YAAAAAAAAAAA%27%2Cgacm:%271~AAAAAAAAAAACAABAAAgAIAAABAAhAAAACAAAAAAAQAAQAAAAAAABBBAAIAAAAAAAAAAAAQAAAIBAAAAAIgMAAAAAAAgAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAgAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAQAAAAAAFAAAABAAAAAAAAAAAARBAAAAAAAAAACAAABAAAAAAAAAAEAAAAAAABAAAAAEAAAAAAAAAAAAAAAAAAACBAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAQAAAAAAAAAAAAAAAACAgBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAgAAAAAAAEAAAAAAAQAAgAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAkQAAAAAAAAAAAAAAAAQ=%27%2Cregion:%27dk%27}; modal_page_counter_cookie=1; utag_main=v_id:0180ccb62e2b002302377014532c05054001000f00fbe$_fd:2$_se:1$_ss:1$_st:1652709110390$dc_visit:2$ses_id:1652707310390%3Bexp-session$_pn:1%3Bexp-session$dc_event:1%3Bexp-session; user_obj=eyJ0b2tlbiI6ImQ1YWFhYWI5NzQyZTc2M2FlZjEyYzhjNDBhNGE3NWRhODA3MmZhOTc4MDNkZDdhMWJkNmRlNzYzODRhMjM4MDgiLCJzdWJzY3JpcHRpb25fYXJlYXMiOnsiMCI6IjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSIsIjNhMGYxODU3LWMzOTMtNGI4Ny1hNjIwLTE4YzEyMTA2OWMdfiI6IjNhMGYxODU3LWMzOTMtNGI4Ny1hNjIwLTE4YzEyMTA2OWM3ZiIsIjNhMDg2OGEzLThiZWEtNGE1Mi1hMGUwLTI4OWE0ZTNkMzIxMSI6IjNhMDg2OGEzLThiZWEtNGE1Mi1hMGUwLTI4OWE0ZTNkMzIxMSIsIjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSI6IjlkYjk2OTcxLTFiNmItNDM0NC05OWFiLTg5YTVjM2NkYmU5MSIsIjE0ZmU1ZDI0LWQxYTYtNGNmNy04NmNhLTgzZTlhNDgzNDkIJGkdf4jE0ZmU1ZDI0LWQxYTYtNGNmNy04NmNhLTgzZTlhNDgzNDk5OCIsIjcwZjI1NjRjLTI2ZTItNDU5Yy05MmQ2LTMwOTUwMWQ4MTc1OSI6IjcwZjI1NjRjLTI2ZTItNDU5Yy05MmQ2LTMwOTUwMWQ4MTc1OSIsIjc3ZDgzMGZlLTYwNzYtNDRjMy05MTZmLTMwNzUwODNkZDg1MiI6Ijc3ZDgzMGZlLTYwNzYtNDRjMy05MTZmLTMwNzUwODNkZDg1MiIsIjMxNjNiY2RhLTIxYWEtNDhlZS1hYmFlLTI2NjAzZTA3YjU4NSI6IjMxNjNiY2RhLT5YWEtNDhlZS1hYmFlLTI2NjAzZTA3YjU4NSIsIjQwMzcyMzEzLTZkYTItNDJhZi04NzBlLWRmMGE5MDU2MmZmNiI6IjQwMzcyMzEzLTZkYTItNDJhZi04NzBlLWRmMGE5MDU2MmZmNiIsImE3ZGQzMjc2LTJiNDMtNDg2NS1iNGViLTFiZTEwMDk3NmQ5NiI6ImE3ZGQzMjc2LTJiNDMtNDg2NS1iNGViLTFiZTEwMDk3NmQ5NiIsImUxY2ExNzc2LTU0NTUtNDc4MC04OTMzLTgxM2M0NjlmZmZhMSI6ImUxY2ExNzc2LTU0NTUtNDc4MC071TMzLTgxM2M0NjlmZmZhMSIsImY2ZWQ5NmVhLWU4MjAtNDg3My1iMmM2LTcwYzRjODc1ZjUyMCI6ImY2ZWQ5NmVhLWU4MjAtNDg3My1iMmM2LTcwYzRjODc1ZjUyMCIsImY3YjViYjRjLTZmYWUtNDdhYi05YTgzLWUyNGVjMzllZWY1ZSI6ImY3YjViYjRjLTZmYWUtNDdhYi05YTgzLWUyNGVjMzllZWY1ZSJ9LCJlbWFpbCI6InpvZCtzdWJzY3JpcHRpb250eXBlMkBwZXl0ei5kayIsInN1YnNjcmlwdGlvbl90eXBlIjoiMiJ9; user_token=d5aaaab9742e763aef12c8c40a4a75da8072fa97803dd7a1bd6de76384a23808; wordpress_test_cookie=WP%20Cookie%20check; wordpress_logged_in_a8d2d12312312379974dc3f916b356d3=MYWORDPRESSUSERNAME%7C1612378456%7C8xaOuvvv7N7Aq4PzHreiIfjKNoRt6NInOxbwvWW9nYL%7Cc70240d7f46d0dc5b6112554da95f69cee61180ebb2b3a93326cb42d02937f3f
Pragma: no-cache
Cache-Control: no-cache
Response headers
HTTP/1.1 400 Bad Request
Date: Mon, 16 May 2022 13:27:53 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Length: 299
Connection: close
Content-Type: text/html; charset=iso-8859-1
Response
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at vagrant Port 80</address>
</body></html>
I can't see out of that, why that request was 'bad'. Nor why the server couldn't understand it.
Solution attempt 3: Check the logs
I checked these:
/var/log/apache2/access.log
/var/log/apache2/error.log
/var/log/apache2/other_vhosts_access.log
I even tried only outputting lines containing the word 400
, with this command:
tail -f access.log error.log other_vhosts_access.log | awk '/400/'
And I could get them (from the other_vhost_access.log
-file). But I can't see any reason for it to be 'bad' either:
In Firefox
vagrant:80 123.123.123.1 - - [16/May/2022:15:27:53 +0200] "GET /wp/wp-admin/js/common.min.js HTTP/1.1" 400 481 "http://mydomain.test/wp/wp-admin/, http://mydomain.test/wp/wp-admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0, Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:101.0) Gecko/20100101 Firefox/101.0"
In Chrome
vagrant:80 123.123.123.1 - - [16/May/2022:15:45:24 +0200] "POST /es/query/ HTTP/1.1" 400 481 "http://mydomain.test/, http://mydomain.test/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36, Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36"
Solution attempt 4: Change LogLevel on the Apache-server
As advised here I tried Changing the LogLevel on the Apache-server.
But I didn't see any better logs in access.log
, error.log
nor other_vhost_access.log
.
Solution attempt 5: Debug using tcpdump
I also tried running tcpdump
from inside the box and output it all using the command:
13:01:42.102438 IP (tos 0x0, ttl 64, id 54239, offset 0, flags [DF], proto TCP (6), length 52)
123.123.123.12.80 > 123.123.123.4.57340: Flags [.], cksum 0xc3db (incorrect -> 0xe5ed), seq 114948251, ack 2535382125, win 505, options [nop,nop,TS val 243067183 ecr 1641029207], length 0
13:01:42.102475 IP (tos 0x0, ttl 64, id 8630, offset 0, flags [DF], proto TCP (6), length 52)
123.123.123.12.80 > 123.123.123.4.57334: Flags [.], cksum 0xc3db (incorrect -> 0xbf8a), seq 2046057008, ack 256413570, win 505, options [nop,nop,TS val 243067183 ecr 3257787508], length 0
13:01:42.106114 IP (tos 0x2,ECT(0), ttl 64, id 54240, offset 0, flags [DF], proto TCP (6), length 533)
123.123.123.12.80 > 123.123.123.4.57340: Flags [P.], cksum 0xc5bc (incorrect -> 0xe068), seq 114948251:114948732, ack 2535382125, win 505, options [nop,nop,TS val 243067186 ecr 1641029207], length 481: HTTP, length: 481
HTTP/1.1 400 Bad Request
Date: Mon, 16 May 2022 11:01:42 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Length: 299
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at vagrant Port 80</address>
</body></html>
13:01:42.106394 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 52)
123.123.123.4.57340 > 123.123.123.12.80: Flags [.], cksum 0xddfb (correct), seq 2535382125, ack 114948732, win 2051, options [nop,nop,TS val 1641029211 ecr 243067186], length 0
13:01:42.109623 IP (tos 0x0, ttl 64, id 54241, offset 0, flags [DF], proto TCP (6), length 52)
123.123.123.12.80 > 123.123.123.4.57340: Flags [F.], cksum 0xc3db (incorrect -> 0xe400), seq 114948732, ack 2535382125, win 505, options [nop,nop,TS val 243067190 ecr 1641029211], length 0
And again, I can see the 400 Bad request
-error. But I can't see what's bad about it.
Solution attempt 6: Look into headers / cookies
Since the errors only occurred when I was logged in, then I tried looking at the header and cookies. But I couldn't see anything spectacular there.
I even compared them to the headers/cookies on another machine, where this is running flawlessly, and nothing sprung in my eye.
... And still... Even if it was this, then why don't I get a better error message?
Solution attempt 7: Server performance - activity monitor
I checked the disk-space, CPU-usage and memory usage on the server (via bpytop) when I refreshed a page. But it looks normal.
Untried attempts
- Downgrade and reinstall and/or upgrade WordPress core.
- Stop and start apache
- Restart the Ubuntu-server.
I've not done this, since I would really like to find out, how to see what is wrong here (instead of just trying to fix it).