You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-09 - 21:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Mar 09, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
332399260,25cyclictest20695-21cron11:05:012
3328992523,1cyclictest606-21dbus-daemon08:10:003
3323992523,1cyclictest27272-21ssh11:11:122
332399250,24cyclictest0-21swapper/212:03:102
332399250,24cyclictest0-21swapper/209:42:132
332399250,1cyclictest29905-21diskmemload12:08:092
331999250,25cyclictest0-21swapper/111:19:341
331999250,25cyclictest0-21swapper/109:21:301
331999250,24cyclictest0-21swapper/110:04:451
331999250,24cyclictest0-21swapper/110:04:451
3317992523,1cyclictest0-21swapper/010:03:220
3317992523,1cyclictest0-21swapper/010:03:220
3328992422,1cyclictest0-21swapper/311:58:193
332899240,24cyclictest29905-21diskmemload10:16:593
332899240,24cyclictest0-21swapper/311:35:493
332899240,23cyclictest0-21swapper/312:25:013
332899240,0cyclictest29905-21diskmemload09:42:043
3323992424,0cyclictest0-21swapper/209:57:062
3323992423,0cyclictest29905-21diskmemload11:16:292
3323992422,1cyclictest0-21swapper/209:22:352
3323992422,0cyclictest0-21swapper/212:34:212
3323992421,2cyclictest17698-21ssh09:29:072
332399241,22cyclictest19572-21basename11:35:012
332399240,24cyclictest0-21swapper/211:35:042
332399240,23cyclictest29905-21diskmemload12:37:392
332399240,23cyclictest29905-21diskmemload10:29:302
332399240,23cyclictest27875-21idleruntime-cro08:00:002
3319992424,0cyclictest0-21swapper/112:26:431
3319992424,0cyclictest0-21swapper/110:59:231
3319992424,0cyclictest0-21swapper/110:26:341
3319992423,0cyclictest0-21swapper/109:43:341
3319992422,1cyclictest0-21swapper/109:49:041
331999240,23cyclictest0-21swapper/111:48:451
331999240,23cyclictest0-21swapper/111:41:571
331999240,23cyclictest0-21swapper/111:11:071
331999240,23cyclictest0-21swapper/111:05:281
331999240,23cyclictest0-21swapper/110:46:341
331999240,23cyclictest0-21swapper/110:05:351
331799243,21cyclictest62550irq/127-eno109:22:160
3317992423,1cyclictest30732-21ssh10:43:150
3317992423,1cyclictest29905-21diskmemload11:00:320
3317992423,1cyclictest0-21swapper/009:39:520
3317992422,1cyclictest25733-21sh11:41:190
3317992422,1cyclictest18447-21sh11:33:230
3317992422,1cyclictest17801-21cron07:40:010
3317992422,1cyclictest0-21swapper/012:23:270
331799240,2cyclictest23043-21ssh10:05:450
331799240,24cyclictest0-21swapper/010:50:260
331799240,23cyclictest0-21swapper/009:52:140
331799240,1cyclictest23109-21ntp_states11:06:410
331799240,0cyclictest29905-21diskmemload12:27:170
331799240,0cyclictest29905-21diskmemload10:33:290
331799240,0cyclictest0-21swapper/012:31:200
3328992323,0cyclictest29905-21diskmemload12:08:043
3328992323,0cyclictest29905-21diskmemload09:27:203
3328992323,0cyclictest20304-21apt-get08:51:333
3328992323,0cyclictest0-21swapper/311:30:263
3328992323,0cyclictest0-21swapper/311:22:473
3328992323,0cyclictest0-21swapper/310:45:163
3328992323,0cyclictest0-21swapper/310:02:103
3328992323,0cyclictest0-21swapper/310:02:103
3328992321,1cyclictest9045-21ssh10:53:093
3328992321,1cyclictest0-21swapper/309:46:373
3328992320,2cyclictest6029-21ssh10:20:263
3328992320,2cyclictest24381-21rm12:10:573
332899230,23cyclictest29905-21diskmemload11:28:423
332899230,23cyclictest29905-21diskmemload10:25:133
332899230,23cyclictest29905-21diskmemload09:10:503
332899230,22cyclictest30458-21ssh11:45:303
332899230,22cyclictest29905-21diskmemload10:35:513
332899230,22cyclictest24015-21ssh11:07:013
332899230,1cyclictest9155-21apt-get09:21:333
332899230,1cyclictest606-21dbus-daemon07:50:013
332899230,14cyclictest0-21swapper/309:19:433
332899230,0cyclictest29905-21diskmemload10:08:243
332899230,0cyclictest0-21swapper/311:15:063
332899230,0cyclictest0-21swapper/310:43:143
3323992323,0cyclictest0-21swapper/212:12:162
3323992323,0cyclictest0-21swapper/211:27:122
3323992323,0cyclictest0-21swapper/211:05:042
3323992323,0cyclictest0-21swapper/209:45:562
3323992323,0cyclictest0-21swapper/209:19:542
3323992322,1cyclictest29905-21diskmemload11:41:032
3323992321,1cyclictest7052-21ssh09:50:152
3323992320,2cyclictest6055-21ssh10:51:092
3323992319,4cyclictest0-21swapper/210:20:322
332399230,2cyclictest30471-21ssh11:45:312
332399230,2cyclictest29905-21diskmemload10:46:362
332399230,2cyclictest14431-21ssh10:58:252
332399230,23cyclictest0-21swapper/210:19:452
332399230,22cyclictest7823-21ssh12:25:472
332399230,22cyclictest29905-21diskmemload09:39:332
332399230,22cyclictest28490-21ssh10:11:132
332399230,22cyclictest19934-21latency_hist08:50:002
332399230,1cyclictest22342-21munin-run10:05:012
332399230,1cyclictest22342-21munin-run10:05:012
332399230,0cyclictest0-21swapper/208:26:342
3319992323,0cyclictest29905-21diskmemload10:40:461
3319992323,0cyclictest0-21swapper/112:16:591
3319992323,0cyclictest0-21swapper/111:28:491
3319992323,0cyclictest0-21swapper/111:01:261
3319992323,0cyclictest0-21swapper/110:36:041
3319992323,0cyclictest0-21swapper/110:11:151
3319992323,0cyclictest0-21swapper/109:30:231
3319992322,0cyclictest0-21swapper/111:22:571
3319992321,1cyclictest0-21swapper/112:11:531
3319992321,1cyclictest0-21swapper/112:02:401
3319992321,1cyclictest0-21swapper/111:50:451
3319992321,1cyclictest0-21swapper/109:29:391
331999230,23cyclictest0-21swapper/111:37:011
331999230,22cyclictest2925-21mailstats10:16:411
331999230,22cyclictest28782-21munin-run09:40:001
331999230,22cyclictest27531-21dump-pmu-power09:05:011
331999230,22cyclictest10400-21perf10:55:011
331999230,22cyclictest0-21swapper/109:12:151
331999230,1cyclictest0-21swapper/110:22:571
331999230,0cyclictest0-21swapper/112:21:191
331999230,0cyclictest0-21swapper/112:09:221
331999230,0cyclictest0-21swapper/111:32:261
331999230,0cyclictest0-21swapper/109:18:211
3317992323,0cyclictest0-21swapper/012:39:060
3317992323,0cyclictest0-21swapper/010:38:200
3317992323,0cyclictest0-21swapper/010:16:550
3317992322,1cyclictest29905-21diskmemload11:51:320
3317992322,1cyclictest29905-21diskmemload10:45:020
3317992321,2cyclictest75-21kswapd010:26:330
3317992321,1cyclictest0-21swapper/012:15:020
3317992321,1cyclictest0-21swapper/012:05:350
3317992321,1cyclictest0-21swapper/011:25:320
3317992321,1cyclictest0-21swapper/011:16:440
331799230,2cyclictest30751-21sh11:45:560
331799230,23cyclictest0-21swapper/010:57:280
331799230,22cyclictest29628-21wc10:11:380
331799230,22cyclictest10526-21latency_hist07:25:000
331799230,22cyclictest0-21swapper/009:13:450
331799230,0cyclictest29905-21diskmemload11:35:360
331799230,0cyclictest0-21swapper/011:58:220
331799230,0cyclictest0-21swapper/009:45:370
3328992222,0cyclictest0-21swapper/311:42:183
3328992222,0cyclictest0-21swapper/309:52:303
3328992221,1cyclictest16404-21ssh10:30:213
3328992221,1cyclictest16298-21ssh11:00:473
332899220,22cyclictest0-21swapper/309:36:553
332899220,21cyclictest12753-21ssh12:30:243
332899220,21cyclictest10572-21sh12:27:243
332899220,21cyclictest0-21swapper/312:15:193
332899220,21cyclictest0-21swapper/310:56:103
332899220,1cyclictest17874-21ssh12:35:153
332899220,1cyclictest0-21swapper/309:55:553
3323992222,0cyclictest29905-21diskmemload11:21:202
3323992220,1cyclictest29905-21diskmemload09:30:072
3323992220,1cyclictest0-21swapper/210:30:202
3323992219,2cyclictest3125-21ssh12:21:332
332399220,22cyclictest0-21swapper/208:06:342
332399220,21cyclictest9740-21smartctl11:56:362
332399220,21cyclictest31138-21apt-get09:11:332
332399220,21cyclictest29905-21diskmemload10:35:072
332399220,21cyclictest28663-21apt-get08:01:352
332399220,21cyclictest28633-21hddtemp_smartct10:41:382
332399220,21cyclictest0-21swapper/210:05:152
332399220,1cyclictest32696-21ssh12:17:372
3319992222,0cyclictest0-21swapper/112:30:441
3319992221,1cyclictest7080-21ssh09:50:171
3319992221,1cyclictest0-21swapper/108:41:341
331999220,1cyclictest27893-21perf08:00:001
331999220,1cyclictest18409-21sh12:36:151
331999220,1cyclictest16436-21rm10:30:241
331999220,0cyclictest29905-21diskmemload11:55:301
3317992222,0cyclictest0-21swapper/009:25:550
3317992221,1cyclictest62550irq/127-eno109:16:560
3317992221,1cyclictest0-21swapper/012:10:310
3317992220,1cyclictest10259-21cron08:30:010
331799221,21cyclictest62550irq/127-eno112:00:120
331799220,21cyclictest6761-21sed11:21:410
331799220,21cyclictest14858-21kworker/u8:211:10:090
331799220,21cyclictest0-21swapper/009:41:430
3328992120,1cyclictest3156-21ssh11:50:273
332899210,0cyclictest0-21swapper/312:00:163
332899210,0cyclictest0-21swapper/310:11:063
3323992119,1cyclictest5463-21turbostat08:20:012
3323992118,2cyclictest25078-21idleruntime-cro07:55:002
332399210,20cyclictest0-21swapper/208:11:322
3319992121,0cyclictest0-21swapper/108:21:311
3319992119,2cyclictest22695-21latency_hist07:50:011
331999210,1cyclictest10702-21apt-get08:31:331
331999210,0cyclictest0-21swapper/107:36:331
3317992121,0cyclictest0-21swapper/008:16:330
3317992120,1cyclictest0-21swapper/009:30:050
3317992120,0cyclictest0-21swapper/007:26:330
3317992119,1cyclictest27880-21apt-get08:00:000
331799210,20cyclictest0-21swapper/008:36:330
331799210,0cyclictest0-21swapper/008:11:320
3328992020,0cyclictest0-21swapper/309:06:333
3328992020,0cyclictest0-21swapper/308:31:333
3328992020,0cyclictest0-21swapper/308:26:333
3328992020,0cyclictest0-21swapper/308:00:023
3323992020,0cyclictest0-21swapper/208:55:022
3323992019,0cyclictest5858-21apt-get08:21:312
3323992017,2cyclictest1-21systemd07:30:012
332399200,0cyclictest0-21swapper/207:31:342
3319992020,0cyclictest0-21swapper/108:56:341
331999200,0cyclictest0-21swapper/107:51:351
331999200,0cyclictest0-21swapper/107:31:341
3317992020,0cyclictest0-21swapper/008:46:340
3317992020,0cyclictest0-21swapper/008:21:310
3317992020,0cyclictest0-21swapper/007:46:320
3317992018,1cyclictest22304-21idleruntime-cro08:55:000
331799200,0cyclictest0-21swapper/009:06:330
3328991919,0cyclictest6043-21apt-get07:16:333
3328991919,0cyclictest22325-21apt-get08:55:023
3328991919,0cyclictest0-21swapper/307:41:333
3323991919,0cyclictest25514-21apt-get09:01:352
3323991919,0cyclictest0-21swapper/207:21:332
3323991918,1cyclictest0-21swapper/208:41:342
3323991917,1cyclictest3648-21apt-get07:11:332
3319991918,0cyclictest0-21swapper/107:41:321
3317991919,0cyclictest0-21swapper/009:01:350
3317991918,1cyclictest0-21swapper/007:11:330
3317991918,0cyclictest0-21swapper/008:56:340
3328991818,0cyclictest0-21swapper/308:36:333
3328991818,0cyclictest0-21swapper/308:25:013
3328991818,0cyclictest0-21swapper/307:55:023
3328991818,0cyclictest0-21swapper/307:31:343
3328991818,0cyclictest0-21swapper/307:26:343
3328991817,1cyclictest3440-21apt-get08:16:333
3328991817,1cyclictest0-21swapper/307:51:353
3323991818,0cyclictest0-21swapper/207:16:332
3323991817,1cyclictest0-21swapper/207:36:332
3323991817,0cyclictest27906-21apt-get09:06:332
3323991815,2cyclictest22688-21latency_hist07:50:012
332399180,0cyclictest0-21swapper/208:31:332
3319991818,0cyclictest0-21swapper/107:11:331
3317991818,0cyclictest0-21swapper/008:06:340
3317991816,1cyclictest27880-21apt-get08:00:020
3328991717,0cyclictest1024-21apt-get08:11:323
3323991716,0cyclictest0-21swapper/208:51:332
3319991716,1cyclictest8253-21apt-get08:26:341
331799170,16cyclictest15516-21apt-get08:41:340
3328991616,0cyclictest0-21swapper/307:21:333
3319991615,0cyclictest0-21swapper/108:51:331
3317991616,0cyclictest0-21swapper/008:31:330
3317991614,1cyclictest23065-21apt-get07:51:350
3328991515,0cyclictest0-21swapper/308:41:343
3328991515,0cyclictest0-21swapper/307:11:333
3319991515,0cyclictest0-21swapper/109:06:331
3319991515,0cyclictest0-21swapper/108:11:321
3319991513,1cyclictest0-21swapper/108:00:021
331999150,1cyclictest0-21swapper/107:26:341
3328991414,0cyclictest0-21swapper/308:45:313
332899140,14cyclictest0-21swapper/309:00:523
332899140,14cyclictest0-21swapper/307:35:543
332399140,14cyclictest0-21swapper/207:40:332
332399140,0cyclictest0-21swapper/208:35:262
3319991414,0cyclictest0-21swapper/108:45:151
3319991414,0cyclictest0-21swapper/108:35:031
3319991414,0cyclictest0-21swapper/108:15:021
3319991414,0cyclictest0-21swapper/108:05:131
3319991414,0cyclictest0-21swapper/107:16:131
3317991414,0cyclictest0-21swapper/007:40:410
331799140,14cyclictest0-21swapper/007:30:490
331799140,0cyclictest0-21swapper/007:15:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional