You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 05:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Aug 03, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3592162995951,5cyclictest3883838-21latency_hist23:50:011
3592168995350,2cyclictest3695592-21kworker/u8:1+flush-259:021:35:203
3592166995246,4cyclictest335557-21systemd-journal23:40:012
3592158994842,4cyclictest3623235-21latency_hist19:40:000
3592168994743,3cyclictest3576836-21kworker/u8:3+flush-259:020:15:543
3592168994643,2cyclictest3685358-21kworker/u8:2+events_unbound20:45:003
3592158994640,4cyclictest3644251-21latency_hist20:00:000
3592158994441,2cyclictest3716356-21kworker/u8:3+events_unbound23:15:000
3592168994339,2cyclictest3695592-21kworker/u8:1+events_unbound22:20:193
3592166994338,3cyclictest3695592-21kworker/u8:1+events_unbound21:35:222
3592166994337,4cyclictest509-21rs:main2
3592166994337,4cyclictest3695926-21latency_hist20:50:012
3592162994336,5cyclictest3847324-21latency_hist23:15:011
3592168994138,2cyclictest3884074-21kworker/u8:4+events_unbound00:10:533
3592168994137,3cyclictest3884074-21kworker/u8:4+events_unbound00:23:173
3592168994137,3cyclictest3638448-21kworker/u8:0+events_unbound20:54:543
3592166994037,2cyclictest3716356-21kworker/u8:3+events_unbound22:25:232
3592166994036,3cyclictest3716872-21apt-get21:10:112
3592162994034,4cyclictest335557-21systemd-journal21:26:051
3592158994036,3cyclictest3607674-21kworker/u8:1+flush-259:019:45:540
3592168993936,2cyclictest3716356-21kworker/u8:3+events_unbound22:51:563
3592168993935,3cyclictest3576836-21kworker/u8:3+events_unbound20:23:553
3592158993936,2cyclictest3842204-21kworker/u8:1+events_unbound23:25:010
3592158993936,2cyclictest3716356-21kworker/u8:3+events_unbound22:26:550
3592158993936,2cyclictest3695592-21kworker/u8:1+events_unbound21:29:550
3592158993936,2cyclictest3695592-21kworker/u8:1+events_unbound21:23:540
3592168993835,2cyclictest3795326-21kworker/u8:2+events_unbound22:37:553
3592168993831,2cyclictest3654569-21kworker/u8:1+events_unbound20:10:203
3592166993834,3cyclictest3894052-21kworker/u8:1+events_unbound00:04:532
3592168993734,2cyclictest3842204-21kworker/u8:1+flush-259:023:26:543
3592168993734,2cyclictest3437287-21kworker/u8:0+events_unbound19:20:383
3592168993733,3cyclictest3904602-21kworker/u8:0+events_unbound00:29:583
3592168993733,3cyclictest3716356-21kworker/u8:3+events_unbound22:01:563
3592168993733,3cyclictest3695592-21kworker/u8:1+events_unbound21:08:553
3592166993734,2cyclictest3904602-21kworker/u8:0+events_unbound00:15:252
3592166993733,3cyclictest3904602-21kworker/u8:0+events_unbound00:30:542
3592162993731,4cyclictest3780024-21cat22:10:001
3592158993734,2cyclictest3904602-21kworker/u8:0+events_unbound00:37:540
3592158993734,2cyclictest3716356-21kworker/u8:3+events_unbound22:58:550
3592158993734,2cyclictest3576836-21kworker/u8:3+events_unbound19:14:540
3592158993733,3cyclictest3878918-21kworker/u8:2+events_unbound23:53:550
3592168993633,2cyclictest3716356-21kworker/u8:3+events_unbound22:27:543
3592168993633,2cyclictest3716356-21kworker/u8:3+events_unbound21:14:533
3592168993633,2cyclictest3638448-21kworker/u8:0+events_unbound20:33:533
3592166993632,3cyclictest3795326-21kworker/u8:2+events_unbound23:07:542
3592166993632,3cyclictest3607674-21kworker/u8:1+flush-259:019:50:542
3592166993630,4cyclictest3862893-21gpgv23:30:012
3592162993630,4cyclictest335557-21systemd-journal21:45:001
3592158993633,2cyclictest3685358-21kworker/u8:2+events_unbound20:45:560
3592158993632,3cyclictest3695592-21kworker/u8:1+events_unbound22:24:550
3592168993532,2cyclictest3716356-21kworker/u8:3+flush-259:023:01:533
3592168993532,2cyclictest3685358-21kworker/u8:2+events_unbound20:45:573
3592168993532,2cyclictest3576836-21kworker/u8:3+flush-259:020:25:243
3592168993532,2cyclictest3576836-21kworker/u8:3+events_unbound19:45:223
3592158993532,2cyclictest3716356-21kworker/u8:3+events_unbound21:19:530
3592158993532,2cyclictest3437287-21kworker/u8:0+events_unbound19:28:540
3592168993431,2cyclictest3716356-21kworker/u8:3+events_unbound22:40:543
3592168993431,2cyclictest3716356-21kworker/u8:3+events_unbound22:31:533
3592168993431,2cyclictest3607674-21kworker/u8:1+events_unbound19:51:533
3592168993430,3cyclictest3695592-21kworker/u8:1+events_unbound22:10:123
3592168993430,3cyclictest3437287-21kworker/u8:0+events_unbound19:15:143
3592166993430,3cyclictest3638448-21kworker/u8:0+flush-259:020:20:552
3592166993429,3cyclictest3716356-21kworker/u8:3+flush-259:023:15:212
3592158993431,2cyclictest3716356-21kworker/u8:3+events_unbound22:41:200
3592158993431,2cyclictest3576836-21kworker/u8:3+events_unbound20:25:530
3592158993430,3cyclictest3716356-21kworker/u8:3+events_unbound22:30:210
3592158993430,3cyclictest3607674-21kworker/u8:1+events_unbound19:54:550
3592168993330,2cyclictest3576836-21kworker/u8:3+events_unbound20:05:503
3592166993329,3cyclictest3884074-21kworker/u8:4+events_unbound00:09:542
3592166993327,4cyclictest335557-21systemd-journal00:00:012
3592162993328,3cyclictest335557-21systemd-journal21:50:011
3592158993330,2cyclictest3857465-21kworker/u8:0+events_unbound23:45:230
3592158993330,2cyclictest3842204-21kworker/u8:1+flush-259:023:40:140
3592158993330,2cyclictest3716356-21kworker/u8:3+events_unbound23:20:230
3592158993330,2cyclictest3716356-21kworker/u8:3+events_unbound23:02:540
3592158993330,2cyclictest3607674-21kworker/u8:1+events_unbound19:32:550
3592168993229,2cyclictest3857465-21kworker/u8:0+events_unbound23:45:533
3592168993229,2cyclictest3716356-21kworker/u8:3+events_unbound23:14:543
3592168993229,2cyclictest3701129-21kworker/u8:2+flush-259:021:00:253
3592168993229,2cyclictest3695592-21kworker/u8:1+events_unbound21:57:543
3592166993229,2cyclictest3695592-21kworker/u8:1+events_unbound21:10:002
3592166993228,3cyclictest3795326-21kworker/u8:2+events_unbound22:57:542
3592166993228,3cyclictest3638448-21kworker/u8:0+events_unbound20:40:202
3592158993229,2cyclictest3695592-21kworker/u8:1+events_unbound20:59:540
3592168993127,3cyclictest3716356-21kworker/u8:3+events_unbound21:25:153
3592166993128,2cyclictest3925295-21kworker/u8:2+events_unbound00:38:262
3592166993128,2cyclictest3701129-21kworker/u8:2+events_unbound21:00:232
3592166993128,2cyclictest3437287-21kworker/u8:0+events_unbound19:10:282
3592166993127,3cyclictest3857465-21kworker/u8:0+events_unbound23:51:542
3592166993127,3cyclictest3716356-21kworker/u8:3+events_unbound22:35:542
3592166993127,3cyclictest3576836-21kworker/u8:3+events_unbound19:19:542
3592162993125,4cyclictest3841283-21ntpq23:05:221
3592162993125,4cyclictest335557-21systemd-journal20:24:521
3592162993124,5cyclictest335557-21systemd-journal19:40:531
3592158993129,2cyclictest3695592-21kworker/u8:1+events_unbound21:44:540
3592158993125,2cyclictest3716356-21kworker/u8:3+events_unbound23:05:180
3592158993124,2cyclictest3701129-21kworker/u8:2+events_unbound21:00:180
3592168993027,2cyclictest3716356-21kworker/u8:3+flush-259:023:05:193
3592168993027,2cyclictest3716356-21kworker/u8:3+events_unbound22:45:223
3592168993027,2cyclictest3716356-21kworker/u8:3+events_unbound22:45:213
3592166993027,2cyclictest3904602-21kworker/u8:0+events_unbound00:29:542
3592166993027,2cyclictest3884074-21kworker/u8:4+events_unbound00:10:212
3592166993026,3cyclictest3695592-21kworker/u8:1+events_unbound21:50:542
3592162993024,4cyclictest3700197-21ntpq20:50:201
3592162993023,5cyclictest3715727-21ntpq21:05:221
3592158993028,2cyclictest3576836-21kworker/u8:3+events_unbound20:22:540
3592158993027,2cyclictest3695592-21kworker/u8:1+events_unbound21:31:040
3592158993027,2cyclictest3638448-21kworker/u8:0+flush-259:020:35:150
3592168992926,2cyclictest3857465-21kworker/u8:0+events_unbound23:30:243
3592168992926,2cyclictest3716356-21kworker/u8:3+events_unbound21:30:213
3592168992925,3cyclictest3695592-21kworker/u8:1+events_unbound22:15:193
3592168992924,4cyclictest3618027-21latency_hist19:35:003
3592166992926,2cyclictest3695592-21kworker/u8:1+flush-259:022:15:242
3592166992925,3cyclictest3716356-21kworker/u8:3+events_unbound22:01:122
3592166992925,3cyclictest3695592-21kworker/u8:1+flush-259:022:10:212
3592166992925,3cyclictest3695592-21kworker/u8:1+events_unbound20:50:582
3592158992926,2cyclictest3716356-21kworker/u8:3+events_unbound22:50:190
3592166992825,2cyclictest3857465-21kworker/u8:0+events_unbound23:23:542
3592166992825,2cyclictest3644017-21kworker/u8:2+events_unbound20:14:542
3592166992825,2cyclictest3576836-21kworker/u8:3+events_unbound20:25:152
3592166992824,3cyclictest3716356-21kworker/u8:3+flush-259:021:40:132
3592166992824,3cyclictest3716356-21kworker/u8:3+events_unbound22:23:542
3592166992824,3cyclictest3638448-21kworker/u8:0+events_unbound20:31:542
3592166992824,3cyclictest3576836-21kworker/u8:3+events_unbound20:19:542
3592166992824,3cyclictest3576836-21kworker/u8:3+events_unbound19:48:542
3592166992824,3cyclictest3576836-21kworker/u8:3+events_unbound19:36:552
3592166992823,4cyclictest3695592-21kworker/u8:1+events_unbound21:31:562
3592162992823,3cyclictest3800745-21apt-get22:30:121
3592162992822,4cyclictest3716356-21kworker/u8:3+flush-259:022:19:591
3592162992821,5cyclictest3863462-21apt-get23:30:101
3592158992825,2cyclictest3716356-21kworker/u8:3+events_unbound22:35:160
3592158992825,2cyclictest3659747-21kworker/u8:4+events_unbound20:16:540
3592166992724,2cyclictest3716356-21kworker/u8:3+flush-259:021:55:162
3592166992724,2cyclictest3716356-21kworker/u8:3+events_unbound23:10:242
3592166992723,3cyclictest3857465-21kworker/u8:0+flush-259:023:30:032
3592166992723,3cyclictest3857465-21kworker/u8:0+events_unbound23:46:532
3592166992722,3cyclictest3732075-21latency_hist21:25:012
3592166992721,2cyclictest3716356-21kworker/u8:3+events_unbound22:45:202
3592166992721,2cyclictest3716356-21kworker/u8:3+events_unbound22:45:192
3592158992724,2cyclictest3904602-21kworker/u8:0+events_unbound00:15:170
3592158992724,2cyclictest3716356-21kworker/u8:3+events_unbound22:05:170
3592168992623,2cyclictest3857465-21kworker/u8:0+events_unbound23:50:163
3592166992623,2cyclictest3695592-21kworker/u8:1+flush-259:021:15:142
3592166992622,3cyclictest3716356-21kworker/u8:3+events_unbound23:00:132
3592166992622,3cyclictest3716356-21kworker/u8:3+events_unbound22:30:532
3592166992622,3cyclictest3695592-21kworker/u8:1+events_unbound22:08:552
3592162992622,3cyclictest3685358-21kworker/u8:2+flush-259:020:45:221
3592158992623,2cyclictest3716356-21kworker/u8:3+events_unbound23:15:280
3592168992522,2cyclictest3904602-21kworker/u8:0+events_unbound00:35:193
3592168992522,2cyclictest3644017-21kworker/u8:2+events_unbound20:00:253
3592168992522,2cyclictest3607674-21kworker/u8:1+events_unbound19:35:213
3592168992518,5cyclictest3757928-21latency_hist21:50:013
3592166992522,2cyclictest3716356-21kworker/u8:3+events_unbound21:25:272
3592166992522,2cyclictest3695592-21kworker/u8:1+events_unbound21:45:232
3592166992521,3cyclictest3795326-21kworker/u8:2+events_unbound22:41:552
3592166992521,3cyclictest3716356-21kworker/u8:3+events_unbound22:51:262
3592162992520,3cyclictest3629031-21apt-get19:45:121
3592162992519,4cyclictest335557-21systemd-journal23:30:011
3592158992519,5cyclictest3649430-21latency_hist20:05:010
3592168992421,2cyclictest3842204-21kworker/u8:1+flush-259:023:35:183
3592166992420,3cyclictest3607674-21kworker/u8:1+events_unbound19:40:102
3592162992419,3cyclictest3716356-21kworker/u8:3+flush-259:021:15:011
3592162992416,5cyclictest3675302-21latency_hist20:30:001
3592158992421,2cyclictest3644017-21kworker/u8:2+events_unbound20:05:140
3592158992421,2cyclictest3638448-21kworker/u8:0+events_unbound20:30:280
3592158992420,3cyclictest3701129-21kworker/u8:2+events_unbound21:05:120
3592168992319,3cyclictest3695592-21kworker/u8:1+events_unbound21:20:113
3592166992319,3cyclictest3437287-21kworker/u8:0+events_unbound19:24:542
3592162992318,4cyclictest3592372-21apt-get19:10:121
3592162992318,3cyclictest3716356-21kworker/u8:3+flush-259:022:50:201
3592162992317,4cyclictest3884074-21kworker/u8:4+flush-259:000:05:011
3592162992317,4cyclictest3769069-21cp22:00:011
3592162992317,4cyclictest335557-21systemd-journal22:36:531
3592166992219,2cyclictest3884074-21kworker/u8:4+events_unbound00:20:192
3592162992215,5cyclictest335557-21systemd-journal23:16:531
3592158992219,2cyclictest3894433-21kworker/u8:3+flush-259:000:05:220
3592158992215,3cyclictest3576836-21kworker/u8:3+events_unbound19:16:530
3592168992119,2cyclictest3894433-21kworker/u8:3+events_unbound00:05:183
3592168992118,2cyclictest3695592-21kworker/u8:1+events_unbound22:07:543
3592166992115,4cyclictest509-21rs:main2
3592162992116,3cyclictest3716356-21kworker/u8:3+flush-259:021:40:001
3592162992115,4cyclictest3831681-21/usr/sbin/munin23:00:151
359216299211,3cyclictest3904574-21munin-run00:10:001
3592168992017,2cyclictest3716356-21kworker/u8:3+flush-259:021:40:243
3592166992017,2cyclictest3857465-21kworker/u8:0+events_unbound23:41:372
3592158992013,1cyclictest191rcu_preempt22:00:000
3592168991915,3cyclictest3644243-21sh20:00:013
3592168991914,4cyclictest335557-21systemd-journal19:30:013
3592162991913,4cyclictest3576836-21kworker/u8:3+flush-259:020:20:011
3592162991912,5cyclictest3779414-21apt-get22:10:001
3592158991917,1cyclictest0-21swapper/019:45:000
3592158991911,1cyclictest191rcu_preempt00:25:180
3592168991816,2cyclictest3638448-21kworker/u8:0+flush-259:020:35:213
3592168991815,2cyclictest3904602-21kworker/u8:0+events_unbound00:15:293
3592168991815,2cyclictest3623493-21apt-get19:40:103
3592162991814,3cyclictest3716356-21kworker/u8:3+flush-259:022:55:231
3592162991814,3cyclictest3716356-21kworker/u8:3+flush-259:022:40:231
3592162991812,4cyclictest3576836-21kworker/u8:3+flush-259:019:35:001
3592158991815,2cyclictest3904602-21kworker/u8:0+events_unbound00:30:260
3592158991815,2cyclictest3795326-21kworker/u8:2+events_unbound22:46:250
3592158991815,2cyclictest3795326-21kworker/u8:2+events_unbound22:46:250
3592166991714,2cyclictest3644017-21kworker/u8:2+events_unbound20:00:232
3592166991714,2cyclictest3576836-21kworker/u8:3+events_unbound20:36:562
359216299179,6cyclictest3705443-21ntpq20:55:231
3592162991714,2cyclictest3857465-21kworker/u8:0+events_unbound23:40:201
3592162991714,2cyclictest3711438-21cron21:05:011
3592162991712,3cyclictest3644017-21kworker/u8:2+flush-259:020:05:001
3592162991711,4cyclictest335557-21systemd-journal23:39:541
3592168991613,2cyclictest3716356-21kworker/u8:3+events_unbound23:15:173
3592168991613,2cyclictest3695592-21kworker/u8:1+events_unbound21:51:083
3592166991613,2cyclictest3695592-21kworker/u8:1+events_unbound20:55:212
359216299165,6cyclictest3930476-21munin-run00:35:011
3592158991614,2cyclictest3644017-21kworker/u8:2+flush-259:020:10:530
3592158991614,1cyclictest60350irq/132-enp0s29f1:rx-021:50:160
3592158991614,1cyclictest3690788-21sh20:45:010
3592158991613,2cyclictest60350irq/132-enp0s29f1:rx-023:35:170
3592158991613,2cyclictest60350irq/132-enp0s29f1:rx-021:15:010
3592158991613,2cyclictest3773857-21unixbench_singl22:00:260
3592158991612,3cyclictest3638448-21kworker/u8:0+events_unbound20:54:350
359216899157,3cyclictest0-21swapper/323:20:163
3592168991512,2cyclictest3826630-21cpuspeed_turbos22:55:143
3592166991512,2cyclictest3576836-21kworker/u8:3+flush-259:019:55:202
3592166991511,3cyclictest3576836-21kworker/u8:3+events_unbound19:30:132
3592162991511,3cyclictest3686293-21irqcore20:40:161
3592162991511,2cyclictest0-21swapper/122:45:191
3592162991511,2cyclictest0-21swapper/122:45:191
3592158991513,1cyclictest60350irq/132-enp0s29f1:rx-022:18:540
3592158991513,1cyclictest60350irq/132-enp0s29f1:rx-019:20:150
3592158991512,1cyclictest3784992-21turbostat22:14:590
359216899145,2cyclictest191rcu_preempt19:10:003
3592168991413,1cyclictest3695592-21kworker/u8:1+events_unbound20:55:153
359216299148,4cyclictest335557-21systemd-journal20:12:531
359216299147,5cyclictest490-21dbus-daemon21:49:591
3592162991411,2cyclictest500-21systemd-logind19:25:011
3592162991411,2cyclictest3917492-21kthreadcore00:20:171
3592158991412,1cyclictest60350irq/132-enp0s29f1:rx-023:34:470
3592158991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:150
3592158991412,1cyclictest60350irq/132-enp0s29f1:rx-021:35:480
3592158991411,2cyclictest3884074-21kworker/u8:4+events_unbound00:10:100
359216899138,3cyclictest3878671-21cron23:45:003
359216899135,1cyclictest191rcu_preempt00:00:013
3592168991310,2cyclictest3722158-21cpuspeed_turbos21:15:133
359216299137,4cyclictest335557-21systemd-journal00:15:001
3592162991310,2cyclictest3620614-21kthreadcore19:35:171
3592158991311,1cyclictest60350irq/132-enp0s29f1:rx-023:56:500
3592158991311,1cyclictest60350irq/132-enp0s29f1:rx-000:00:180
359216899128,3cyclictest3884074-21kworker/u8:4+events_unbound00:02:333
359216299129,2cyclictest3885127-21kthreadcore23:50:181
359216299128,3cyclictest3857465-21kworker/u8:0+flush-259:023:56:241
359216299128,2cyclictest2971-21cron21:20:011
359216299128,2cyclictest0-21swapper/119:20:001
359216299127,4cyclictest3576836-21kworker/u8:3+flush-259:019:50:171
359216299121,2cyclictest3790831-21ps22:20:171
359216899111,2cyclictest191rcu_preempt00:30:213
359216299118,2cyclictest3795793-21fschecks_time22:25:151
359216299118,2cyclictest3737611-21cpuspeed_turbos21:30:141
359216299118,2cyclictest0-21swapper/100:35:221
359216299117,3cyclictest3576836-21kworker/u8:3+flush-259:019:55:251
359216299107,2cyclictest3924314-21ntpq00:25:201
359216299107,2cyclictest3910229-21fschecks_count00:15:161
359216299107,2cyclictest3727430-21fschecks_time21:20:141
359216299107,2cyclictest0-21swapper/120:35:191
359216299105,4cyclictest3576836-21kworker/u8:3+flush-259:019:25:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional