You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 01:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Wed May 06, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2265799260,25cyclictest17710-21ssh12:30:301
2265399260,25cyclictest938-21latency_hist09:25:000
2266499250,24cyclictest16058-21diskmemload11:46:413
2266499250,24cyclictest16058-21diskmemload10:38:323
2266499250,24cyclictest0-21swapper/310:10:513
2266099250,24cyclictest683-21basename10:50:012
2266099250,0cyclictest0-21swapper/210:59:432
2265799250,24cyclictest16058-21diskmemload12:25:081
2265799250,1cyclictest0-21swapper/110:59:481
2265399250,1cyclictest16058-21diskmemload10:46:210
22664992422,1cyclictest0-21swapper/309:27:243
2266499240,23cyclictest16058-21diskmemload11:23:263
2266099240,23cyclictest16058-21diskmemload11:22:072
2266099240,23cyclictest0-21swapper/212:06:262
2266099240,23cyclictest0-21swapper/211:36:592
2266099240,1cyclictest21808-21awk10:40:012
22657992422,1cyclictest0-21swapper/110:10:001
22657992421,2cyclictest7670-21sh11:52:231
2265799240,24cyclictest0-21swapper/110:31:081
2265799240,23cyclictest16058-21diskmemload12:15:011
2265799240,23cyclictest16058-21diskmemload09:15:161
2265799240,0cyclictest0-21swapper/110:43:441
22653992422,1cyclictest27768-21ssh09:47:400
2265399240,24cyclictest21521-21ssh09:41:440
2265399240,23cyclictest0-21swapper/010:06:110
2265399240,1cyclictest16058-21diskmemload10:15:420
2265399240,0cyclictest16058-21diskmemload11:26:170
2265399240,0cyclictest16058-21diskmemload09:27:590
22664992323,0cyclictest16058-21diskmemload12:11:323
22664992323,0cyclictest0-21swapper/311:53:283
22664992323,0cyclictest0-21swapper/310:45:233
22664992322,1cyclictest9311-21sh10:00:463
22664992322,1cyclictest16058-21diskmemload11:11:363
22664992322,0cyclictest0-21swapper/311:55:223
22664992321,1cyclictest0-21swapper/312:27:463
22664992321,1cyclictest0-21swapper/310:54:383
22664992321,1cyclictest0-21swapper/310:31:483
22664992321,1cyclictest0-21swapper/309:59:213
22664992321,1cyclictest0-21swapper/309:41:423
2266499230,2cyclictest6910-21ssh12:21:063
2266499230,23cyclictest0-21swapper/312:30:273
2266499230,23cyclictest0-21swapper/310:05:393
2266499230,23cyclictest0-21swapper/309:13:183
2266499230,22cyclictest17743-21uname12:01:323
2266499230,22cyclictest16058-21diskmemload12:06:443
2266499230,22cyclictest16058-21diskmemload11:36:273
2266499230,22cyclictest16058-21diskmemload11:18:193
2266499230,22cyclictest14340-21sh11:30:293
2266499230,22cyclictest10094-21ntp_kernel_pll_11:26:313
2266499230,0cyclictest16058-21diskmemload09:45:163
2266499230,0cyclictest0-21swapper/309:50:243
22660992323,0cyclictest16058-21diskmemload09:49:282
22660992323,0cyclictest0-21swapper/212:35:022
22660992323,0cyclictest0-21swapper/212:02:082
22660992323,0cyclictest0-21swapper/210:11:232
22660992322,1cyclictest16058-21diskmemload09:34:262
22660992321,1cyclictest0-21swapper/210:44:422
22660992319,4cyclictest0-21swapper/211:40:252
2266099230,2cyclictest16058-21diskmemload10:20:012
2266099230,23cyclictest0-21swapper/211:55:282
2266099230,23cyclictest0-21swapper/210:31:192
2266099230,23cyclictest0-21swapper/209:40:512
2266099230,22cyclictest32235-21sh12:14:272
2266099230,22cyclictest16058-21diskmemload12:15:452
2266099230,22cyclictest16058-21diskmemload10:09:452
2266099230,22cyclictest15796-21sh09:36:372
2266099230,22cyclictest0-21swapper/210:23:262
2266099230,22cyclictest0-21swapper/209:25:082
2266099230,22cyclictest0-21swapper/209:16:372
2266099230,1cyclictest0-21swapper/211:09:312
2266099230,0cyclictest16058-21diskmemload12:25:292
2266099230,0cyclictest0-21swapper/209:55:412
22657992323,0cyclictest0-21swapper/111:37:141
22657992323,0cyclictest0-21swapper/110:46:201
22657992323,0cyclictest0-21swapper/110:23:591
22657992323,0cyclictest0-21swapper/110:10:201
22657992323,0cyclictest0-21swapper/109:23:121
22657992321,1cyclictest19764-21sh10:37:121
22657992321,1cyclictest0-21swapper/112:06:471
22657992320,2cyclictest1555-21rm11:19:211
2265799230,23cyclictest16058-21diskmemload11:22:181
2265799230,23cyclictest16058-21diskmemload09:33:411
2265799230,23cyclictest0-21swapper/111:56:571
2265799230,23cyclictest0-21swapper/109:12:431
2265799230,22cyclictest27195-21ssh09:46:471
2265799230,22cyclictest0-21swapper/109:42:191
2265799230,1cyclictest3972-21sh12:17:311
2265799230,0cyclictest0-21swapper/112:22:431
2265799230,0cyclictest0-21swapper/112:02:031
2265799230,0cyclictest0-21swapper/111:41:571
2265799230,0cyclictest0-21swapper/111:25:451
22653992323,0cyclictest62550irq/127-eno111:55:010
22653992323,0cyclictest16058-21diskmemload11:00:430
22653992323,0cyclictest16058-21diskmemload10:40:320
22653992323,0cyclictest0-21swapper/012:10:050
22653992323,0cyclictest0-21swapper/012:02:440
22653992322,1cyclictest16058-21diskmemload10:20:170
2265399230,2cyclictest24421-21sendmail08:20:010
2265399230,23cyclictest18066-21rm11:33:070
2265399230,23cyclictest0-21swapper/011:53:040
2265399230,23cyclictest0-21swapper/009:30:080
2265399230,22cyclictest9637-21ssh10:56:470
2265399230,22cyclictest5482-21ssh10:25:380
2265399230,22cyclictest21617-21ssh11:07:470
2265399230,22cyclictest16437-21rm10:35:040
2265399230,22cyclictest0-21swapper/012:36:520
2265399230,0cyclictest16058-21diskmemload11:15:340
2265399230,0cyclictest0-21swapper/012:27:270
2265399230,0cyclictest0-21swapper/010:04:540
2265399230,0cyclictest0-21swapper/009:58:590
22664992222,0cyclictest18228-21kworker/u8:112:35:083
22664992222,0cyclictest16058-21diskmemload12:15:443
22664992222,0cyclictest0-21swapper/311:03:003
22664992222,0cyclictest0-21swapper/310:55:333
22664992220,1cyclictest0-21swapper/311:40:033
22664992220,1cyclictest0-21swapper/309:31:313
22664992219,2cyclictest16058-21diskmemload09:20:033
2266499220,22cyclictest0-21swapper/311:05:113
2266499220,22cyclictest0-21swapper/310:16:133
2266499220,21cyclictest0-21swapper/310:41:393
2266499220,21cyclictest0-21swapper/310:20:533
2266499220,1cyclictest0-21swapper/309:36:253
2266499220,1cyclictest0-21swapper/309:15:193
2266499220,1cyclictest0-21swapper/308:41:253
22660992222,0cyclictest0-21swapper/212:30:122
22660992222,0cyclictest0-21swapper/211:30:122
22660992222,0cyclictest0-21swapper/211:26:022
22660992222,0cyclictest0-21swapper/210:00:352
22660992221,1cyclictest16058-21diskmemload10:29:502
22660992220,1cyclictest0-21swapper/210:50:092
2266099220,21cyclictest4158-21diskmemload11:50:062
2266099220,21cyclictest4036-21perf08:45:002
2266099220,21cyclictest16058-21diskmemload09:20:182
2266099220,21cyclictest11532-21munin-plugin-st12:25:012
2266099220,21cyclictest0-21swapper/211:48:092
2266099220,21cyclictest0-21swapper/209:50:192
2266099220,21cyclictest0-21swapper/209:06:262
2266099220,21cyclictest0-21swapper/207:46:252
2266099220,21cyclictest0-21swapper/207:26:262
2266099220,1cyclictest0-21swapper/211:15:422
2266099220,1cyclictest0-21swapper/211:10:282
2266099220,1cyclictest0-21swapper/209:11:292
22657992222,0cyclictest16058-21diskmemload10:25:541
22657992222,0cyclictest0-21swapper/111:07:501
22657992222,0cyclictest0-21swapper/109:35:251
22657992222,0cyclictest0-21swapper/109:28:031
22657992220,1cyclictest28186-21apt-get07:21:251
22657992220,1cyclictest0-21swapper/110:54:011
22657992220,1cyclictest0-21swapper/109:50:071
2265799220,22cyclictest7724-21ssh09:58:581
2265799220,21cyclictest14656-21ssh11:30:531
2265799220,21cyclictest0-21swapper/110:18:301
2265799220,1cyclictest24149-21ssh12:36:131
22653992222,0cyclictest0-21swapper/012:20:330
22653992222,0cyclictest0-21swapper/012:18:510
22653992222,0cyclictest0-21swapper/011:46:490
22653992221,1cyclictest0-21swapper/010:30:500
22653992220,1cyclictest2363-21sh11:20:010
2265399220,22cyclictest0-21swapper/012:33:010
2265399220,22cyclictest0-21swapper/011:12:450
2265399220,22cyclictest0-21swapper/010:14:540
2265399220,22cyclictest0-21swapper/009:51:560
2265399220,21cyclictest22728-21ssh09:15:390
2265399220,21cyclictest22564-21apt-get07:10:020
2265399220,21cyclictest0-21swapper/008:11:260
2265399220,21cyclictest0-21swapper/008:11:260
2265399220,21cyclictest0-21swapper/007:16:260
2265399220,1cyclictest26256-21ssh11:41:210
2265399220,1cyclictest0-21swapper/009:11:060
2265399220,1cyclictest0-21swapper/009:00:010
22664992121,0cyclictest519-21apt-get07:31:253
22664992121,0cyclictest0-21swapper/308:26:263
22664992120,1cyclictest0-21swapper/307:10:023
22660992121,0cyclictest0-21swapper/207:21:252
22660992120,0cyclictest24803-21apt-get08:21:252
2266099210,20cyclictest0-21swapper/207:36:252
22657992120,1cyclictest7937-21apt-get07:46:261
22657992120,1cyclictest0-21swapper/111:00:091
2265799210,21cyclictest0-21swapper/111:45:111
2265799210,20cyclictest0-21swapper/110:00:201
2265799210,0cyclictest9237-21apt-get08:56:261
22653992121,0cyclictest30584-21apt-get07:26:260
22653992120,1cyclictest30196-21cron07:25:010
2265399210,20cyclictest0-21swapper/007:46:260
2265399210,0cyclictest0-21swapper/011:35:340
2265399210,0cyclictest0-21swapper/010:50:200
22664992019,0cyclictest15142-21apt-get08:01:253
22664992018,1cyclictest8886-21grep08:55:003
2266499200,0cyclictest0-21swapper/307:26:263
22660992018,1cyclictest10330-21apt-get07:51:262
2266099200,20cyclictest0-21swapper/208:16:342
2266099200,0cyclictest29594-21apt-get08:31:262
22657992020,0cyclictest0-21swapper/108:21:261
22657992020,0cyclictest0-21swapper/107:51:251
22657992018,1cyclictest22327-21apt-get08:16:261
2265799200,0cyclictest0-21swapper/107:26:261
22653992020,0cyclictest5483-21apt-get07:41:250
22653992020,0cyclictest0-21swapper/008:50:000
2265399200,20cyclictest0-21swapper/008:01:250
22664991919,0cyclictest0-21swapper/309:01:263
22664991919,0cyclictest0-21swapper/308:46:263
22664991919,0cyclictest0-21swapper/308:21:253
22664991919,0cyclictest0-21swapper/307:41:253
22660991919,0cyclictest0-21swapper/207:56:262
22660991917,1cyclictest27775-21cron07:20:002
22657991919,0cyclictest0-21swapper/108:46:261
22657991919,0cyclictest0-21swapper/107:15:001
22657991917,1cyclictest29230-21basename08:30:011
22653991919,0cyclictest0-21swapper/009:01:260
22653991919,0cyclictest0-21swapper/008:31:260
22653991919,0cyclictest0-21swapper/007:56:260
22664991818,0cyclictest0-21swapper/308:56:263
2266499180,17cyclictest31633-21perf08:35:003
22660991818,0cyclictest0-21swapper/208:01:242
22660991816,1cyclictest2606-21ls07:35:012
22657991818,0cyclictest0-21swapper/108:06:251
22657991818,0cyclictest0-21swapper/107:31:251
22657991717,0cyclictest0-21swapper/107:36:251
22657991714,2cyclictest31617-21idleruntime-cro08:35:001
22653991717,0cyclictest0-21swapper/008:36:260
22653991717,0cyclictest0-21swapper/008:06:250
22660991616,0cyclictest4392-21apt-get08:46:262
22660991615,1cyclictest17535-21apt-get08:06:252
22660991614,1cyclictest0-21swapper/208:15:012
22660991614,1cyclictest0-21swapper/208:15:012
2266099160,15cyclictest0-21swapper/209:00:012
22657991616,0cyclictest0-21swapper/108:41:251
22657991614,1cyclictest7584-21idleruntime-cro07:45:001
2265799160,15cyclictest0-21swapper/107:20:011
2265799160,11cyclictest622-21gdbus08:00:001
22653991616,0cyclictest0-21swapper/009:06:260
22653991614,1cyclictest0-21swapper/007:51:350
22653991613,2cyclictest2584-21sh07:35:010
22664991515,0cyclictest14047-21apt-get09:06:263
22664991515,0cyclictest0-21swapper/308:16:263
22664991515,0cyclictest0-21swapper/307:51:253
22664991513,1cyclictest1615-21apt08:40:003
22664991513,1cyclictest10025-21munin-run07:50:003
22660991513,1cyclictest0-21swapper/207:15:002
2266099150,1cyclictest783-21snmpd07:43:012
2265799153,11cyclictest605-21avahi-daemon09:00:331
22657991515,0cyclictest0-21swapper/109:06:261
2265399154,9cyclictest611-21irqbalance08:54:080
22664991414,0cyclictest0-21swapper/307:35:323
22664991414,0cyclictest0-21swapper/307:15:093
2266499140,14cyclictest0-21swapper/308:10:343
2266499140,14cyclictest0-21swapper/308:10:343
2266499140,14cyclictest0-21swapper/307:56:493
2266499140,14cyclictest0-21swapper/307:20:073
2266499140,0cyclictest0-21swapper/308:05:083
22660991414,0cyclictest0-21swapper/208:50:462
2266099140,14cyclictest0-21swapper/209:01:152
2266099140,14cyclictest0-21swapper/208:35:032
2266099140,14cyclictest0-21swapper/208:26:402
2265799140,0cyclictest0-21swapper/108:10:201
2265799140,0cyclictest0-21swapper/108:10:201
22653991414,0cyclictest0-21swapper/008:41:080
22653991414,0cyclictest0-21swapper/008:25:050
22653991414,0cyclictest0-21swapper/008:20:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional