You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 04:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue May 05, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
150742241163,25sleep20-21swapper/219:08:412
151602236163,23sleep10-21swapper/119:09:501
150572223192,21sleep00-21swapper/019:08:280
151372204168,24sleep30-21swapper/319:09:313
2158821520,2sleep21526899cyclictest19:20:202
296521490,1sleep20-21swapper/219:50:272
1258021350,4sleep01525499cyclictest20:15:120
15268999390,2cyclictest0-21swapper/220:52:272
15268998884,3cyclictest0-21swapper/221:26:392
15268998784,2cyclictest0-21swapper/200:14:232
15268997471,2cyclictest0-21swapper/220:05:302
15268996966,2cyclictest0-21swapper/223:32:432
15268995652,3cyclictest0-21swapper/219:17:452
1526299361,14cyclictest28754-21kworker/u8:220:52:371
1527499331,2cyclictest0-21swapper/321:27:113
174322320,5sleep0909-21sssd_nss00:05:240
1527499321,20cyclictest0-21swapper/300:21:593
1527499311,14cyclictest0-21swapper/322:03:393
15262993112,12cyclictest0-21swapper/120:40:531
1526299311,12cyclictest0-21swapper/120:35:251
1526299311,11cyclictest0-21swapper/100:21:301
1525499318,11cyclictest25967-21basename22:00:010
1527499301,15cyclictest0-21swapper/320:27:503
1527499301,14cyclictest0-21swapper/320:57:563
1527499301,14cyclictest0-21swapper/320:51:323
1527499301,13cyclictest0-21swapper/323:44:323
1527499301,13cyclictest0-21swapper/322:06:073
1527499301,12cyclictest0-21swapper/321:06:273
1527499301,11cyclictest0-21swapper/319:38:513
1527499300,15cyclictest0-21swapper/319:41:193
1527499300,13cyclictest0-21swapper/320:24:443
1526299301,6cyclictest0-21swapper/122:33:571
1526299301,12cyclictest0-21swapper/119:34:261
1526299301,11cyclictest15670-21open_inodes20:20:211
1526299301,11cyclictest0-21swapper/123:40:451
1525499305,12cyclictest31922-21munin-run21:00:020
15254993010,16cyclictest22826-21munin-run00:20:010
101942300,3sleep30-21swapper/323:50:133
1527499291,13cyclictest0-21swapper/322:22:393
1527499291,13cyclictest0-21swapper/319:26:433
1527499291,13cyclictest0-21swapper/300:28:353
1527499291,12cyclictest0-21swapper/323:58:193
1527499291,12cyclictest0-21swapper/323:58:193
1527499291,12cyclictest0-21swapper/323:48:073
1527499291,12cyclictest0-21swapper/322:59:283
1527499291,12cyclictest0-21swapper/320:41:483
1527499291,12cyclictest0-21swapper/320:13:163
1527499291,12cyclictest0-21swapper/300:36:393
1527499291,11cyclictest0-21swapper/323:18:523
1527499291,11cyclictest0-21swapper/322:31:313
1527499291,11cyclictest0-21swapper/321:45:473
1527499291,11cyclictest0-21swapper/320:37:523
1527499291,10cyclictest0-21swapper/323:27:343
1526299293,19cyclictest495-21jbd2/dm-0-823:02:011
1526299291,13cyclictest0-21swapper/123:57:141
1526299291,13cyclictest0-21swapper/123:57:131
1526299291,12cyclictest0-21swapper/121:22:141
1526299291,11cyclictest12925-21latency_hist21:30:021
1526299291,11cyclictest0-21swapper/122:06:581
1526299291,11cyclictest0-21swapper/119:24:291
1526299291,11cyclictest0-21swapper/100:15:581
1526299291,10cyclictest0-21swapper/119:39:581
305832280,3sleep1221ktimersoftd/119:40:231
1527499281,24cyclictest0-21swapper/319:47:513
1527499281,12cyclictest0-21swapper/321:18:553
1527499281,12cyclictest0-21swapper/321:14:153
1527499281,12cyclictest0-21swapper/319:21:593
1527499281,12cyclictest0-21swapper/300:30:593
1527499281,11cyclictest0-21swapper/323:09:083
1527499281,11cyclictest0-21swapper/320:48:483
1527499281,11cyclictest0-21swapper/319:52:033
1527499281,11cyclictest0-21swapper/319:34:393
1527499281,10cyclictest0-21swapper/300:19:233
1526299288,9cyclictest12990-21/usr/sbin/munin21:30:221
1526299281,5cyclictest0-21swapper/121:51:151
1526299281,11cyclictest0-21swapper/122:43:011
1526299281,11cyclictest0-21swapper/121:17:491
1526299281,10cyclictest0-21swapper/122:35:311
1525499284,12cyclictest27230-21awk00:30:010
15254992810,15cyclictest26928-21basename19:35:020
1527499271,9cyclictest0-21swapper/323:30:083
1527499271,9cyclictest0-21swapper/322:36:563
1527499271,8cyclictest0-21swapper/322:40:033
1527499271,2cyclictest0-21swapper/300:14:033
1527499271,11cyclictest0-21swapper/323:37:043
1527499271,11cyclictest0-21swapper/321:03:403
1527499271,11cyclictest0-21swapper/300:00:473
1527499271,10cyclictest0-21swapper/322:47:123
1527499271,10cyclictest0-21swapper/300:05:473
1527499270,12cyclictest0-21swapper/321:56:593
1526899271,12cyclictest0-21swapper/219:39:332
1526899271,11cyclictest0-21swapper/221:19:032
1526899271,11cyclictest0-21swapper/219:31:372
1526899271,10cyclictest0-21swapper/223:38:572
1526899271,10cyclictest0-21swapper/222:03:242
1526299271,11cyclictest0-21swapper/120:45:131
1526299271,10cyclictest0-21swapper/123:30:211
15254992712,12cyclictest3977-21munin-run21:10:020
1527499261,12cyclictest0-21swapper/322:53:363
1527499261,12cyclictest0-21swapper/322:53:363
1527499261,12cyclictest0-21swapper/321:35:473
1527499261,11cyclictest0-21swapper/320:15:363
1527499261,10cyclictest0-21swapper/319:19:543
1527499260,11cyclictest0-21swapper/321:41:033
1527499260,11cyclictest0-21swapper/321:41:033
1526899261,14cyclictest0-21swapper/223:55:392
1526899261,14cyclictest0-21swapper/223:55:392
1526899261,10cyclictest0-21swapper/222:23:302
1526299267,7cyclictest19326-21hddtemp_smartct00:10:161
1526299261,17cyclictest22750-21df23:05:151
1526299261,16cyclictest0-21swapper/121:07:371
1526299261,16cyclictest0-21swapper/100:28:261
1526299261,15cyclictest0-21swapper/120:13:051
1526299261,12cyclictest24957-21diskstats23:10:131
1526299261,10cyclictest0-21swapper/121:40:501
1526299261,10cyclictest0-21swapper/121:40:501
1526299261,10cyclictest0-21swapper/120:32:331
1526299261,10cyclictest0-21swapper/120:32:331
1526299261,10cyclictest0-21swapper/119:26:461
1525499261,13cyclictest17448-21idleruntime-cro19:15:000
1527499251,9cyclictest0-21swapper/322:17:553
1527499251,9cyclictest0-21swapper/320:03:133
1527499251,8cyclictest0-21swapper/323:14:403
1527499251,11cyclictest0-21swapper/323:21:383
1527499251,11cyclictest0-21swapper/321:20:513
1526899251,11cyclictest0-21swapper/221:02:052
1526899251,11cyclictest0-21swapper/220:13:182
1526899251,10cyclictest0-21swapper/220:45:532
1526899250,10cyclictest0-21swapper/223:43:112
1526299256,3cyclictest0-21swapper/122:57:371
1526299251,10cyclictest0-21swapper/123:51:411
1526299251,10cyclictest0-21swapper/119:18:491
1525499251,12cyclictest15595-21awk22:50:000
1527499241,8cyclictest0-21swapper/322:29:353
1527499241,8cyclictest0-21swapper/321:32:393
1527499241,11cyclictest0-21swapper/321:50:273
1527499241,10cyclictest0-21swapper/320:33:023
1527499241,10cyclictest0-21swapper/320:33:023
1526899241,10cyclictest0-21swapper/222:46:462
1526899241,10cyclictest0-21swapper/221:49:172
1526899241,10cyclictest0-21swapper/200:20:252
1526299241,9cyclictest0-21swapper/120:29:171
1526299241,21cyclictest0-21swapper/119:10:051
1526299241,13cyclictest0-21swapper/100:09:221
1526299241,10cyclictest0-21swapper/122:17:181
1526299241,10cyclictest0-21swapper/121:10:291
1525499241,6cyclictest0-21swapper/023:01:530
1527499238,3cyclictest0-21swapper/320:06:013
1527499231,9cyclictest0-21swapper/322:11:393
1526299237,4cyclictest0-21swapper/122:47:391
1526299234,3cyclictest0-21swapper/121:48:301
1526299231,6cyclictest0-21swapper/123:19:551
1526299231,5cyclictest0-21swapper/121:39:581
1526299231,16cyclictest0-21swapper/122:50:391
1526299231,16cyclictest0-21swapper/122:50:391
1526299231,16cyclictest0-21swapper/122:14:361
1526299231,12cyclictest0-21swapper/123:35:401
1525499231,5cyclictest0-21swapper/021:16:480
1525499231,4cyclictest16948-21kworker/u8:021:42:250
1525499231,4cyclictest16948-21kworker/u8:021:42:240
1527499221,14cyclictest0-21swapper/319:59:273
1527499220,11cyclictest0-21swapper/319:11:043
1526899224,6cyclictest24320-40dnf23:09:262
1526899224,4cyclictest9461-21sshd23:48:212
1526899221,6cyclictest26310-21ntp_states00:25:222
1526899221,5cyclictest0-21swapper/221:35:422
1526299221,9cyclictest0-21swapper/122:27:341
1526299221,19cyclictest0-21swapper/121:56:351
1526299221,18cyclictest3219-21head22:20:171
1526299221,18cyclictest13055-21sed20:15:161
1526299221,14cyclictest0-21swapper/120:07:421
1526299221,11cyclictest721-21latency_hist23:30:021
1526299221,11cyclictest0-21swapper/123:21:441
1526299221,10cyclictest0-21swapper/122:02:451
1526299220,4cyclictest9587-21kworker/u8:200:03:321
1525499221,5cyclictest0-21swapper/023:36:040
15254992211,3cyclictest87550irq/128-enp1s0-22:05:210
1525499221,12cyclictest20551-21/usr/sbin/munin19:20:200
1527499211,12cyclictest0-21swapper/323:02:483
1526899216,3cyclictest87650irq/129-enp1s0-22:14:222
1526899214,3cyclictest87350irq/126-enp1s0-22:50:162
1526899214,3cyclictest87350irq/126-enp1s0-22:50:162
1526899211,4cyclictest0-21swapper/223:53:152
1526299216,3cyclictest0-21swapper/121:02:091
1526299216,3cyclictest0-21swapper/120:04:101
1526299212,4cyclictest0-21swapper/123:45:051
1526299211,11cyclictest0-21swapper/119:47:301
1526299211,11cyclictest0-21swapper/100:35:061
1526299210,16cyclictest0-21swapper/119:54:371
1526299210,15cyclictest0-21swapper/100:30:261
1525499219,3cyclictest7919-1kworker/u9:022:13:300
1525499216,2cyclictest7919-1kworker/u9:021:45:250
1525499215,14cyclictest3359-21perf19:55:010
1525499211,6cyclictest0-21swapper/021:29:080
1525499211,4cyclictest0-21swapper/023:34:590
15254992112,3cyclictest0-21swapper/023:28:190
157922201,3sleep015837-21smartctl20:20:230
1526899204,3cyclictest87350irq/126-enp1s0-22:20:022
1526299201,14cyclictest0-21swapper/120:55:261
1526299201,10cyclictest5715-21cat20:00:021
1525499209,3cyclictest26178-21/usr/sbin/munin22:00:140
1525499209,3cyclictest24611-21/usr/sbin/munin23:10:130
1525499203,4cyclictest20631-21munin-run00:15:020
1525499201,7cyclictest0-21swapper/022:18:470
1525499201,5cyclictest29109-21munin-run19:40:020
1525499201,16cyclictest6177-21munin-run21:15:020
1525499201,11cyclictest22414-21/usr/sbin/munin23:05:140
1525499201,11cyclictest18354-21/usr/sbin/munin19:15:230
1526899193,4cyclictest26828-21egrep23:15:132
1526899192,7cyclictest6788-21munin-run22:30:012
1526899192,4cyclictest21188-21df00:15:132
1526899192,3cyclictest15874-21taskset00:00:532
1526899191,5cyclictest9206-21/usr/sbin/munin22:35:112
1526899191,5cyclictest24611-21/usr/sbin/munin23:10:172
1526899191,5cyclictest20205-21/usr/sbin/munin23:00:262
1526899191,5cyclictest15568-21cut20:20:192
1526899191,4cyclictest0-21swapper/200:32:252
15268991913,3cyclictest17412-21gltestperf20:25:162
1526899191,2cyclictest0-21swapper/222:05:282
1526899191,13cyclictest0-21swapper/221:59:302
1526899191,11cyclictest0-21swapper/219:25:222
1525499198,3cyclictest18006-21/usr/sbin/munin22:55:240
1525499197,3cyclictest7919-1kworker/u9:021:31:350
1525499193,4cyclictest1971850kworker/u8:300:22:050
1525499192,5cyclictest7919-1kworker/u9:022:27:420
1525499191,9cyclictest0-21swapper/022:22:110
1525499191,7cyclictest0-21swapper/000:39:000
1525499191,5cyclictest0-21swapper/023:42:430
1525499191,10cyclictest808-21snmpd23:46:070
1525499191,10cyclictest10174-21/usr/sbin/munin20:10:180
1526899181,8cyclictest87350irq/126-enp1s0-22:40:122
1526899181,4cyclictest19924-21sed20:30:192
1526899181,4cyclictest19924-21sed20:30:182
1526899181,4cyclictest0-21swapper/222:30:482
1526899181,4cyclictest0-21swapper/200:05:222
15268991814,2cyclictest312982sleep219:45:012
15268991812,3cyclictest827-21runrttasks19:10:232
15268991811,4cyclictest19585-21grep22:55:262
1526899181,13cyclictest0-21swapper/221:53:452
1526899181,12cyclictest0-21swapper/220:44:012
1526899181,12cyclictest0-21swapper/200:39:102
1526899180,5cyclictest1953-21kworker/2:120:19:562
1525499187,3cyclictest87550irq/128-enp1s0-23:55:030
1525499187,3cyclictest87550irq/128-enp1s0-23:55:030
1525499187,3cyclictest808-21snmpd23:51:080
1525499183,2cyclictest87350irq/126-enp1s0-00:30:380
1525499182,3cyclictest0-21swapper/000:00:440
1525499181,9cyclictest29853-21grep19:40:150
1525499181,5cyclictest24063-21kworker/u8:023:21:220
1525499181,4cyclictest7919-1kworker/u9:022:36:000
1525499181,4cyclictest0-21swapper/022:50:490
1525499181,4cyclictest0-21swapper/022:50:490
15254991811,4cyclictest19619-21sed20:30:150
15254991811,4cyclictest19619-21sed20:30:150
1525499181,10cyclictest808-21snmpd21:50:430
1526899171,7cyclictest87350irq/126-enp1s0-23:26:222
1526899171,5cyclictest0-21swapper/223:21:012
1526899171,2cyclictest0-21swapper/221:12:272
1526899171,2cyclictest0-21swapper/221:07:512
15268991712,3cyclictest21691-21grep20:35:142
1526899171,13cyclictest3918-21diskstats19:55:152
1526899171,12cyclictest0-21swapper/221:32:292
1526899171,12cyclictest0-21swapper/220:01:102
15268991710,4cyclictest8593-21/usr/sbin/munin21:20:182
15268991710,4cyclictest17520-21tr21:40:122
15268991710,4cyclictest17520-21tr21:40:122
1525499171,6cyclictest0-21swapper/022:30:590
1525499171,6cyclictest0-21swapper/020:45:170
15254991712,3cyclictest28225-21cut20:50:160
15254991712,3cyclictest12909-21awk22:40:230
1525499171,15cyclictest0-21swapper/020:26:520
1525499171,14cyclictest0-21swapper/020:08:310
15254991711,3cyclictest27673-21cat23:15:180
1525499171,11cyclictest0-21swapper/021:35:210
15268991611,3cyclictest31416-21unixbench-2d20:55:252
1526899161,10cyclictest0-21swapper/219:45:332
1525499161,7cyclictest808-21snmpd20:35:270
1525499161,6cyclictest9687-21sshd21:20:220
1525499161,14cyclictest0-21swapper/021:02:490
15254991611,3cyclictest31535-21/usr/sbin/munin19:45:170
15254991611,3cyclictest23504-21cpuspeed_turbos20:40:120
15254991610,3cyclictest3570-21/usr/sbin/munin19:55:170
1525499151,7cyclictest808-21snmpd20:00:170
1525499150,8cyclictest0-21swapper/019:25:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional