You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 04:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Apr 19, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4022917994337,4cyclictest4168622-21/usr/sbin/munin21:30:221
4022912994137,3cyclictest4106453-21apt-get20:30:100
4022917993732,3cyclictest4090343-21kworker/u8:2+flush-259:020:40:161
4022919993532,2cyclictest26335-21kworker/u8:0+events_unbound23:20:262
4022922993326,5cyclictest4059028-21latency_hist19:45:013
4022922993326,5cyclictest252-21systemd-journal19:20:013
4022912993329,3cyclictest4132032-21kworker/u8:3+flush-259:022:20:180
4022912993329,2cyclictest26335-21kworker/u8:0+flush-259:023:20:200
4022922993125,4cyclictest252-21systemd-journal22:53:533
4022922993024,4cyclictest4152963-21latency_hist21:15:013
4022922993024,4cyclictest4152963-21latency_hist21:15:013
4022922993024,4cyclictest4095996-21base6420:20:013
4022922993024,4cyclictest252-21systemd-journal00:33:373
4022922993024,4cyclictest252-21systemd-journal00:33:363
4022912993027,2cyclictest4158132-21kworker/u8:2+flush-259:021:45:000
4022922992924,4cyclictest140218-21apt-get00:10:113
4022919992926,2cyclictest4132032-21kworker/u8:3+events_unbound21:45:142
4022917992922,4cyclictest4069344-21apt-get19:55:021
4022912992924,4cyclictest21435-21apt-get22:15:120
4022922992822,4cyclictest4194095-21latency_hist21:55:013
4022917992824,3cyclictest10509-21kworker/u8:1+flush-259:022:20:001
4022917992822,5cyclictest252-21systemd-journal23:10:011
4022912992824,3cyclictest57448-21apt-get22:50:100
4022912992824,3cyclictest4117224-21apt-get20:40:110
4022912992822,4cyclictest71738-21ntpq23:00:240
4022919992724,2cyclictest4080074-21kworker/u8:1+flush-259:020:35:152
4022919992723,3cyclictest47148-21apt-get22:40:102
4022917992722,3cyclictest4132032-21kworker/u8:3+flush-259:022:30:151
4022917992721,4cyclictest360-21apt-get21:55:121
4022912992723,3cyclictest150018-21kworker/u8:2+events_unbound00:22:520
4022912992721,4cyclictest14704-21ntpq22:05:220
4022919992623,3cyclictest4080074-21kworker/u8:1+events_unbound20:14:542
4022919992623,2cyclictest26335-21kworker/u8:0+events_unbound22:50:212
4022919992622,3cyclictest4090343-21kworker/u8:2+flush-259:020:20:232
4022919992622,3cyclictest4033279-21kworker/u8:0+events_unbound20:00:162
4022919992622,3cyclictest139979-21kworker/u8:1+events_unbound00:25:282
4022917992620,4cyclictest252-21systemd-journal19:48:551
4022917992620,4cyclictest165973-21/usr/sbin/munin00:35:141
4022912992622,3cyclictest4033279-21kworker/u8:0+events_unbound19:43:550
4022922992519,4cyclictest509-21rs:main3
4022919992522,2cyclictest4122100-21kworker/u8:0+events_unbound21:15:132
4022919992522,2cyclictest150018-21kworker/u8:2+flush-259:000:35:232
4022917992519,4cyclictest4137764-21/usr/sbin/munin21:00:121
4022917992518,5cyclictest20250-21ntpq22:10:191
4022912992522,2cyclictest150018-21kworker/u8:2+events_unbound00:35:210
4022912992521,3cyclictest4059258-21kworker/u8:2+events_unbound19:45:130
4022912992520,4cyclictest4033255-21latency_hist19:20:010
4022919992421,2cyclictest4033279-21kworker/u8:0+flush-259:019:25:172
4022919992420,3cyclictest26335-21kworker/u8:0+events_unbound22:31:542
4022912992421,2cyclictest139979-21kworker/u8:1+flush-259:000:31:080
4022912992421,2cyclictest139979-21kworker/u8:1+flush-259:000:31:080
4022912992420,3cyclictest78469-21kworker/u8:1+flush-259:023:15:220
4022912992420,3cyclictest4132032-21kworker/u8:3+events_unbound21:50:180
4022912992419,4cyclictest4158085-21latency_hist21:20:010
4022912992418,4cyclictest252-21systemd-journal20:20:010
4022912992417,5cyclictest4074070-21ntpq19:55:210
4022922992318,4cyclictest165998-21apt-get00:35:103
4022919992320,3cyclictest3989342-21kworker/u8:1+events_unbound19:10:142
4022919992320,2cyclictest4132032-21kworker/u8:3+flush-259:021:30:152
4022919992320,2cyclictest119563-21kworker/u8:3+flush-259:000:00:142
4022919992319,3cyclictest119563-21kworker/u8:3+flush-259:000:13:542
4022919992318,4cyclictest4132032-21kworker/u8:3+events_unbound21:23:562
4022917992317,4cyclictest119563-21kworker/u8:3+flush-259:023:55:231
4022912992320,2cyclictest78469-21kworker/u8:1+events_unbound23:40:180
4022912992320,2cyclictest4158132-21kworker/u8:2+flush-259:021:55:130
4022912992320,2cyclictest4080074-21kworker/u8:1+events_unbound20:15:210
4022912992320,2cyclictest4033279-21kworker/u8:0+events_unbound19:35:240
4022912992320,2cyclictest119563-21kworker/u8:3+flush-259:000:05:230
4022912992319,3cyclictest4080074-21kworker/u8:1+events_unbound20:37:560
4022912992319,3cyclictest26335-21kworker/u8:0+flush-259:022:45:000
4022912992319,3cyclictest139979-21kworker/u8:1+events_unbound00:27:540
4022919992219,2cyclictest4132032-21kworker/u8:3+events_unbound23:01:542
4022919992219,2cyclictest26335-21kworker/u8:0+flush-259:023:15:532
4022917992218,3cyclictest150018-21kworker/u8:2+flush-259:000:30:011
4022912992220,2cyclictest4132032-21kworker/u8:3+events_unbound22:45:220
4022912992219,2cyclictest4033279-21kworker/u8:0+events_unbound19:20:210
4022912992218,3cyclictest78469-21kworker/u8:1+events_unbound23:26:540
4022912992218,3cyclictest4132032-21kworker/u8:3+events_unbound22:30:170
4022912992218,3cyclictest4122100-21kworker/u8:0+events_unbound21:35:170
4022912992218,3cyclictest4033279-21kworker/u8:0+events_unbound20:05:180
4022912992218,3cyclictest252-21systemd-journal23:55:530
4022922992112,4cyclictest507-21in:imuxsock23:24:593
4022919992118,2cyclictest4132032-21kworker/u8:3+events_unbound21:00:002
4022919992118,2cyclictest119563-21kworker/u8:3+flush-259:023:55:292
4022919992117,3cyclictest4132032-21kworker/u8:3+events_unbound21:43:552
4022919992117,3cyclictest4064431-21apt-get19:50:112
4022919992117,3cyclictest4033279-21kworker/u8:0+flush-259:019:35:212
4022919992117,3cyclictest4033279-21kworker/u8:0+events_unbound20:07:562
4022919992116,4cyclictest252-21systemd-journal23:45:542
4022917992115,4cyclictest4163243-21latency_hist21:25:011
4022917992114,5cyclictest4032356-21ntpq19:15:211
4022912992118,3cyclictest4132032-21kworker/u8:3+events_unbound21:21:550
4022912992117,3cyclictest4132032-21kworker/u8:3+events_unbound21:31:550
4022912992117,3cyclictest4127525-21apt-get20:50:120
4022912992117,3cyclictest150018-21kworker/u8:2+events_unbound00:17:540
4022912992114,5cyclictest4105366-21ntpq20:25:220
4022922992016,3cyclictest62325-21kworker/u8:2+flush-259:023:05:163
4022922992016,3cyclictest507-21in:imuxsock19:25:003
4022922992014,4cyclictest26335-21kworker/u8:0+flush-259:022:25:163
4022922992013,5cyclictest139950-21latency_hist00:10:013
4022919992018,2cyclictest3979059-21kworker/u8:3+events_unbound19:42:342
4022919992017,2cyclictest4132032-21kworker/u8:3+flush-259:021:05:262
4022919992017,2cyclictest4132032-21kworker/u8:3+events_unbound21:50:142
4022919992017,2cyclictest4090343-21kworker/u8:2+flush-259:020:40:182
4022919992017,2cyclictest4090343-21kworker/u8:2+flush-259:020:25:182
4022919992017,2cyclictest4059258-21kworker/u8:2+events_unbound19:47:332
4022919992017,2cyclictest150018-21kworker/u8:2+flush-259:000:20:152
4022919992016,3cyclictest4132032-21kworker/u8:3+events_unbound22:59:552
4022919992016,3cyclictest4033279-21kworker/u8:0+flush-259:019:55:172
4022919992016,2cyclictest1-21systemd21:40:002
4022917992015,3cyclictest252-21systemd-journal00:03:331
4022917992013,5cyclictest10470-21latency_hist22:05:011
4022912992018,2cyclictest26335-21kworker/u8:0+flush-259:023:50:150
4022912992017,2cyclictest62325-21kworker/u8:2+events_unbound23:05:140
4022912992016,3cyclictest4122100-21kworker/u8:0+events_unbound21:07:500
4022912992016,3cyclictest119563-21kworker/u8:3+events_unbound00:11:540
4022922991915,3cyclictest26335-21kworker/u8:0+flush-259:023:25:163
4022922991914,3cyclictest78469-21kworker/u8:1+flush-259:023:40:163
4022919991916,2cyclictest4017981-21kworker/u8:2+events_unbound19:15:182
4022919991916,2cyclictest26335-21kworker/u8:0+flush-259:023:35:142
4022919991915,3cyclictest4132032-21kworker/u8:3+flush-259:022:45:542
4022919991914,4cyclictest36615-21latency_hist22:30:022
4022917991913,4cyclictest4042733-21ntpq19:25:231
4022917991912,5cyclictest4075153-21/usr/sbin/munin20:00:161
4022912991916,2cyclictest4080074-21kworker/u8:1+flush-259:020:15:000
4022912991916,2cyclictest4033279-21kworker/u8:0+events_unbound19:25:260
4022912991916,2cyclictest26335-21kworker/u8:0+events_unbound23:35:180
4022912991915,3cyclictest4090343-21kworker/u8:2+events_unbound20:47:460
4022912991915,3cyclictest4017981-21kworker/u8:2+flush-259:019:10:210
4022912991915,3cyclictest26335-21kworker/u8:0+flush-259:022:55:180
4022919991815,2cyclictest4132032-21kworker/u8:3+flush-259:022:00:162
4022919991815,2cyclictest4132032-21kworker/u8:3+events_unbound22:20:242
4022919991815,2cyclictest4090343-21kworker/u8:2+flush-259:020:35:002
4022919991815,2cyclictest26335-21kworker/u8:0+flush-259:023:30:252
4022919991815,2cyclictest10509-21kworker/u8:1+events_unbound22:20:002
4022919991815,1cyclictest0-21swapper/220:50:002
4022919991814,3cyclictest139979-21kworker/u8:1+events_unbound00:33:552
4022919991814,3cyclictest139979-21kworker/u8:1+events_unbound00:33:542
4022917991815,2cyclictest111603-21kthreadcore23:40:171
4022917991814,3cyclictest3979059-21kworker/u8:3+flush-259:019:41:271
4022917991814,2cyclictest0-21swapper/122:30:001
4022917991813,3cyclictest4033279-21kworker/u8:0+flush-259:020:10:161
4022917991812,4cyclictest4173739-21/usr/sbin/munin21:35:141
4022912991815,2cyclictest4033279-21kworker/u8:0+flush-259:019:54:050
4022912991814,3cyclictest4163509-21apt-get21:25:100
4022912991814,3cyclictest4122100-21kworker/u8:0+events_unbound21:10:130
4022912991814,3cyclictest4122100-21kworker/u8:0+events_unbound21:10:130
4022912991814,3cyclictest4122100-21kworker/u8:0+events_unbound20:57:550
4022922991714,2cyclictest844-21irqcore21:55:163
4022922991714,2cyclictest4069363-21idleruntime-cro19:55:013
4022922991714,1cyclictest0-21swapper/319:50:003
4022922991712,4cyclictest4112045-21apt-get20:35:143
4022922991712,3cyclictest26310-21latency_hist22:20:013
4022919991714,2cyclictest4158132-21kworker/u8:2+events_unbound22:07:542
4022917991714,2cyclictest4095650-21latency_hist20:20:001
4022917991712,4cyclictest3979059-21kworker/u8:3+flush-259:019:30:171
4022917991711,4cyclictest252-21systemd-journal23:13:181
4022912991713,3cyclictest4132032-21kworker/u8:3+flush-259:021:45:250
4022912991713,3cyclictest4033279-21kworker/u8:0+events_unbound19:31:540
4022912991713,3cyclictest3979059-21kworker/u8:3+events_unbound20:00:220
402292299162,4cyclictest4188044-21ntpq21:45:223
4022922991611,3cyclictest124729-21/usr/sbin/munin23:55:163
4022919991614,2cyclictest4080074-21kworker/u8:1+events_unbound20:15:152
4022919991613,2cyclictest4122100-21kworker/u8:0+flush-259:021:00:172
4022919991612,3cyclictest62325-21kworker/u8:2+events_unbound23:08:532
402291799169,5cyclictest252-21systemd-journal19:55:011
4022917991611,4cyclictest139979-21kworker/u8:1+flush-259:000:10:161
4022917991610,5cyclictest88913-21diskstats23:20:151
4022912991614,1cyclictest60350irq/132-enp0s29f1:rx-022:00:110
4022912991614,1cyclictest26335-21kworker/u8:0+events_unbound23:45:130
4022912991613,2cyclictest4132032-21kworker/u8:3+flush-259:021:00:150
402292299154,2cyclictest191rcu_preempt23:30:203
4022922991510,4cyclictest4111345-21apt-get20:35:003
4022922991510,3cyclictest252-21systemd-journal00:24:593
4022919991513,2cyclictest26335-21kworker/u8:0+events_unbound23:54:532
4022919991512,3cyclictest4132032-21kworker/u8:3+events_unbound21:25:022
4022919991512,2cyclictest78469-21kworker/u8:1+flush-259:023:40:232
4022919991512,2cyclictest4033279-21kworker/u8:0+events_unbound19:20:232
402291799159,4cyclictest4132032-21kworker/u8:3+flush-259:021:45:171
402291799158,5cyclictest4080315-21apt-get20:05:131
402291799150,13cyclictest1-21systemd21:10:541
402291799150,13cyclictest1-21systemd21:10:541
4022912991512,2cyclictest60350irq/132-enp0s29f1:rx-023:10:140
4022912991512,2cyclictest60350irq/132-enp0s29f1:rx-022:26:550
4022912991511,3cyclictest4132032-21kworker/u8:3+events_unbound22:10:560
402292299147,3cyclictest0-21swapper/321:09:583
402292299142,8cyclictest4027820-21sshd19:11:543
402292299142,3cyclictest45995-21ld22:35:213
402292299142,2cyclictest31196-21modprobe22:21:533
402292299142,10cyclictest4158085-21latency_hist21:20:003
402292299141,4cyclictest4166432-21cut21:25:173
402292299141,3cyclictest124459-21modprobe23:54:533
4022922991411,2cyclictest5640-21cpuspeed_turbos22:00:133
4022922991411,2cyclictest4162763-21unixbench_multi21:20:253
4022919991411,2cyclictest26335-21kworker/u8:0+events_unbound23:25:132
4022919991410,3cyclictest4158132-21kworker/u8:2+events_unbound21:59:572
402291799149,3cyclictest1-21systemd20:55:541
402291799149,3cyclictest119317-21apt23:50:001
402291799148,4cyclictest52263-21/usr/sbin/munin22:45:141
402291799148,4cyclictest4142954-21apt-get21:05:141
402291799147,5cyclictest26550-21/usr/sbin/munin22:20:201
402291799147,5cyclictest124486-21latency_hist23:55:001
402291799145,6cyclictest252-21systemd-journal22:10:001
4022917991411,2cyclictest0-21swapper/122:40:141
4022912991412,1cyclictest60350irq/132-enp0s29f1:rx-022:35:090
4022912991412,1cyclictest60350irq/132-enp0s29f1:rx-000:00:170
4022912991411,2cyclictest103936-21turbostat23:35:000
402292299139,3cyclictest155662-21apt-get00:25:133
402292299137,4cyclictest4132374-21latency_hist20:55:003
402292299137,4cyclictest252-21systemd-journal19:57:353
402292299132,2cyclictest4172572-21ntpq21:30:203
402292299131,3cyclictest21194-21systemctl22:15:003
402292299131,11cyclictest0-21swapper/300:04:593
4022922991310,2cyclictest78406-21irqcore23:10:183
402291999139,3cyclictest78469-21kworker/u8:1+events_unbound23:10:222
4022919991310,2cyclictest26335-21kworker/u8:0+flush-259:022:35:212
402291799139,3cyclictest4122100-21kworker/u8:0+flush-259:021:26:001
402291799137,4cyclictest4178922-21/usr/sbin/munin21:40:201
4022917991310,2cyclictest4103589-21kthreadcore20:25:181
402292299129,2cyclictest83249-21cpuspeed_turbos23:15:123
402292299123,2cyclictest50-21ksoftirqd/323:40:003
402292299121,3cyclictest50-21ksoftirqd/300:15:203
402292299121,2cyclictest50-21ksoftirqd/323:45:233
402292299121,2cyclictest50-21ksoftirqd/322:45:163
402292299121,2cyclictest50-21ksoftirqd/322:05:163
402292299121,2cyclictest50-21ksoftirqd/320:05:133
402292299121,2cyclictest4126393-21ntpq20:45:213
402292299121,2cyclictest4094622-21ntpq20:15:213
402292299121,2cyclictest40878-21ntpq22:30:203
402292299121,2cyclictest4047883-21ntpq19:30:233
402292299120,7cyclictest66364-21memory22:55:203
402291999129,2cyclictest16528-21fschecks_time22:10:142
4022919991210,1cyclictest4148494-21irqcore21:10:172
4022919991210,1cyclictest4148494-21irqcore21:10:172
402291799129,2cyclictest67901-21fschecks_count23:00:151
402291799129,2cyclictest64853-21kthreadcore22:55:181
402291799128,2cyclictest0-21swapper/122:40:001
402291799127,4cyclictest150018-21kworker/u8:2+flush-259:000:25:001
402291799121,7cyclictest4101036-21munin-run20:25:001
402292299111,7cyclictest50-21ksoftirqd/320:00:183
402292299111,2cyclictest50-21ksoftirqd/321:00:143
402292299110,2cyclictest50-21ksoftirqd/321:40:153
402292299110,2cyclictest4173925-21diskstats21:35:143
402291999118,3cyclictest4033279-21kworker/u8:0+events_unbound19:31:562
402291799118,2cyclictest61842-21unixbench_multi22:50:241
402291799118,2cyclictest4114358-21kthreadcore20:35:201
402291799117,3cyclictest26335-21kworker/u8:0+events_unbound23:25:171
402291799117,2cyclictest4022897-21cyclictest21:15:181
402291799115,4cyclictest4048978-21/usr/sbin/munin19:35:141
402291799110,8cyclictest103937-21latency_hist23:35:001
402292299102,5cyclictest50-21ksoftirqd/320:40:223
402291799107,1cyclictest0-21swapper/100:30:211
402291799107,1cyclictest0-21swapper/100:30:211
402291799101,4cyclictest139947-21cron00:09:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional