You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 18:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Mon Apr 27, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26021992821,4cyclictest19794-21diskmemload12:32:321
26023992623,2cyclictest29727-21taskset09:50:433
26022992623,2cyclictest8238-21munin-plugin-st11:05:002
2602299260,2cyclictest1286-21cron11:30:012
2602299260,2cyclictest1286-21cron11:30:012
2602199260,26cyclictest0-21swapper/111:37:291
2602199260,1cyclictest32128-21sh09:52:541
26023992524,1cyclictest331rcuc/312:37:423
26023992524,1cyclictest331rcuc/312:29:483
2602399250,24cyclictest313-21systemd-journal10:00:013
2602399250,24cyclictest0-21swapper/312:13:433
26022992523,1cyclictest10956-21latency_hist07:45:002
2602199250,24cyclictest19794-21diskmemload12:10:301
2602199250,24cyclictest19794-21diskmemload11:52:451
2602199250,24cyclictest18477-21apt-get08:00:401
2602199250,1cyclictest0-21swapper/111:13:001
2602199250,0cyclictest20795-21apt-get12:20:401
2602099253,21cyclictest62550irq/127-eno111:14:330
26020992523,1cyclictest20276-21rm09:10:210
2602099250,24cyclictest30903-21ssh09:50:540
2602099250,24cyclictest19794-21diskmemload12:34:270
2602399240,23cyclictest13325-21latency_hist11:10:003
2602399240,23cyclictest0-21swapper/312:07:293
2602399240,23cyclictest0-21swapper/311:10:483
26022992424,0cyclictest0-21swapper/209:58:172
26022992421,2cyclictest31350-21ssh12:30:302
2602299240,24cyclictest0-21swapper/212:12:172
2602299240,23cyclictest0-21swapper/212:27:262
2602299240,22cyclictest723-21ssh10:56:172
26021992422,2cyclictest0-21swapper/111:01:431
26021992421,3cyclictest0-21swapper/112:04:421
26021992420,3cyclictest0-21swapper/111:05:191
26021992419,5cyclictest19794-21diskmemload10:55:451
2602199240,24cyclictest0-21swapper/110:50:061
2602199240,24cyclictest0-21swapper/110:21:411
2602199240,23cyclictest0-21swapper/110:47:451
2602199240,23cyclictest0-21swapper/109:56:261
2602199240,21cyclictest0-21swapper/109:39:021
2602199240,1cyclictest5065-21cat09:25:491
2602199240,1cyclictest0-21swapper/109:00:011
2602199240,0cyclictest0-21swapper/111:32:461
2602199240,0cyclictest0-21swapper/111:17:181
2602199240,0cyclictest0-21swapper/110:42:131
2602199240,0cyclictest0-21swapper/110:00:371
2602199240,0cyclictest0-21swapper/109:12:211
26020992422,1cyclictest15531-21sh10:08:370
26020992421,2cyclictest0-21swapper/010:32:480
2602099240,24cyclictest19794-21diskmemload11:34:090
26023992323,0cyclictest19794-21diskmemload10:11:213
26023992323,0cyclictest0-21swapper/311:58:303
26023992323,0cyclictest0-21swapper/308:15:413
2602399232,1cyclictest34-21ksoftirqd/311:51:213
26023992321,1cyclictest0-21swapper/312:33:533
26023992321,1cyclictest0-21swapper/310:54:463
2602399230,23cyclictest34-21ksoftirqd/311:23:013
2602399230,23cyclictest19794-21diskmemload10:20:253
2602399230,23cyclictest0-21swapper/312:15:393
2602399230,23cyclictest0-21swapper/310:59:563
2602399230,23cyclictest0-21swapper/310:45:073
2602399230,23cyclictest0-21swapper/309:25:043
2602399230,22cyclictest23019-21ssh09:44:443
2602399230,22cyclictest22923-21ssh09:11:513
2602399230,22cyclictest17133-21sh11:45:213
2602399230,22cyclictest16186-21ssh10:09:463
2602399230,22cyclictest1583-21ssh11:30:053
2602399230,0cyclictest6717-21ssh11:35:053
2602399230,0cyclictest0-21swapper/310:40:243
2602399230,0cyclictest0-21swapper/309:22:503
26022992323,0cyclictest19794-21diskmemload12:07:312
26022992323,0cyclictest0-21swapper/210:03:472
26022992323,0cyclictest0-21swapper/209:21:192
26022992322,1cyclictest14777-21ssh11:41:562
26022992321,1cyclictest24218-21apt-get11:20:422
26022992321,1cyclictest23928-21apt-get09:45:412
2602299230,2cyclictest6036-21turbostat07:35:012
2602299230,2cyclictest23207-21ssh10:46:412
2602299230,2cyclictest18415-21sh10:42:132
2602299230,23cyclictest0-21swapper/210:32:452
2602299230,23cyclictest0-21swapper/210:06:592
2602299230,22cyclictest544-21ssh09:54:212
2602299230,22cyclictest30377-21ssh11:57:242
2602299230,22cyclictest26361-21rm11:54:182
2602299230,22cyclictest22992-21ssh09:11:582
2602299230,22cyclictest0-21swapper/210:12:292
2602299230,1cyclictest17055-21ssh11:12:542
2602299230,0cyclictest19794-21diskmemload09:38:122
2602299230,0cyclictest0-21swapper/212:39:262
2602299230,0cyclictest0-21swapper/210:23:072
26021992323,0cyclictest0-21swapper/110:31:051
26021992323,0cyclictest0-21swapper/109:32:351
26021992321,1cyclictest0-21swapper/111:25:431
26021992321,1cyclictest0-21swapper/111:25:431
26021992321,1cyclictest0-21swapper/110:10:281
26021992320,3cyclictest19794-21diskmemload12:18:421
2602199230,22cyclictest29102-21apt-get07:15:401
2602199230,22cyclictest19794-21diskmemload10:15:471
2602199230,22cyclictest0-21swapper/112:37:371
2602199230,22cyclictest0-21swapper/111:55:421
2602199230,22cyclictest0-21swapper/110:29:431
2602199230,22cyclictest0-21swapper/109:20:131
2602199230,22cyclictest0-21swapper/109:15:341
2602199230,0cyclictest0-21swapper/112:26:181
2602199230,0cyclictest0-21swapper/111:43:211
2602199230,0cyclictest0-21swapper/111:20:181
26020992323,0cyclictest0-21swapper/012:05:530
26020992323,0cyclictest0-21swapper/011:07:230
26020992323,0cyclictest0-21swapper/010:22:560
26020992322,1cyclictest62550irq/127-eno110:57:110
2602099232,21cyclictest62550irq/127-eno110:48:080
26020992322,1cyclictest0-21swapper/010:01:350
26020992321,1cyclictest5934-21ssh09:26:510
26020992321,1cyclictest26637-21perf10:20:000
26020992321,1cyclictest0-21swapper/012:01:550
26020992321,1cyclictest0-21swapper/011:49:160
26020992321,1cyclictest0-21swapper/011:15:240
26020992321,1cyclictest0-21swapper/010:25:440
2602099230,2cyclictest20735-21ssh12:20:370
2602099230,2cyclictest10359-21rm12:10:230
2602099230,23cyclictest19794-21diskmemload09:38:190
2602099230,23cyclictest0-21swapper/010:39:090
2602099230,23cyclictest0-21swapper/010:39:090
2602099230,22cyclictest8272-21diskmemload12:39:250
2602099230,22cyclictest3606-21ssh11:00:220
2602099230,22cyclictest30306-21ssh09:20:070
2602099230,22cyclictest19794-21diskmemload09:56:380
2602099230,22cyclictest0-21swapper/011:51:400
2602099230,22cyclictest0-21swapper/011:35:130
2602099230,22cyclictest0-21swapper/010:50:380
2602099230,1cyclictest29778-21ssh12:29:010
2602099230,0cyclictest19794-21diskmemload09:45:030
2602099230,0cyclictest0-21swapper/011:21:460
26023992222,0cyclictest331rcuc/310:15:263
26023992222,0cyclictest331rcuc/309:37:183
26023992222,0cyclictest19794-21diskmemload12:22:593
26023992222,0cyclictest19794-21diskmemload11:00:273
26023992222,0cyclictest0-21swapper/312:04:233
26023992221,1cyclictest25562-21ssh09:15:413
26023992220,1cyclictest331rcuc/311:15:413
26023992220,1cyclictest10510-21ssh10:35:263
26023992220,1cyclictest10510-21ssh10:35:263
2602399220,21cyclictest19794-21diskmemload10:00:053
2602399220,21cyclictest19794-21diskmemload09:30:243
2602399220,21cyclictest0-21swapper/308:05:413
26022992221,1cyclictest8127-21apt-get08:45:432
26022992220,2cyclictest9098-21sh11:36:022
26022992220,2cyclictest12036-21ssh11:07:572
26022992220,1cyclictest5192-21ssh09:25:492
26022992220,1cyclictest0-21swapper/210:50:322
26022992220,1cyclictest0-21swapper/209:30:062
26022992219,2cyclictest31216-21sendmail07:20:012
26022992218,4cyclictest0-21swapper/212:02:352
2602299220,22cyclictest0-21swapper/211:45:172
2602299220,22cyclictest0-21swapper/211:15:312
2602299220,22cyclictest0-21swapper/209:05:422
2602299220,21cyclictest22982-21ssh10:15:442
2602299220,21cyclictest10520-21apt-get08:50:422
2602299220,21cyclictest0-21swapper/210:25:162
2602299220,21cyclictest0-21swapper/208:00:402
2602299220,1cyclictest848-21apt-get08:30:402
2602299220,1cyclictest10419-21ssh10:35:172
2602299220,1cyclictest10419-21ssh10:35:172
2602299220,1cyclictest0-21swapper/212:19:182
2602299220,1cyclictest0-21swapper/209:43:322
2602299220,0cyclictest19794-21diskmemload11:30:422
2602299220,0cyclictest0-21swapper/212:21:032
26021992222,0cyclictest0-21swapper/109:45:211
26021992220,1cyclictest8875-21apt-get07:40:401
26021992220,1cyclictest0-21swapper/108:55:011
2602199220,21cyclictest19794-21diskmemload10:05:171
2602199220,21cyclictest0-21swapper/111:45:571
2602199220,21cyclictest0-21swapper/108:40:421
2602199220,21cyclictest0-21swapper/107:20:421
2602199220,1cyclictest0-21swapper/110:35:071
2602199220,1cyclictest0-21swapper/110:35:071
2602199220,1cyclictest0-21swapper/109:05:411
2602199220,1cyclictest0-21swapper/108:20:011
2602199220,1cyclictest0-21swapper/108:10:011
26020992221,1cyclictest0-21swapper/011:27:000
26020992221,1cyclictest0-21swapper/011:27:000
26020992220,1cyclictest9568-21memory09:30:460
2602099220,22cyclictest0-21swapper/008:30:400
2602099220,21cyclictest15326-21sh10:40:020
2602099220,21cyclictest0-21swapper/011:41:010
2602099220,1cyclictest0-21swapper/012:19:150
2602099220,1cyclictest0-21swapper/010:10:380
2602099220,1cyclictest0-21swapper/009:15:530
26023992121,0cyclictest0-21swapper/307:45:403
2602399210,21cyclictest0-21swapper/311:25:093
2602399210,21cyclictest0-21swapper/311:25:093
2602399210,21cyclictest0-21swapper/308:30:403
2602399210,20cyclictest19794-21diskmemload09:45:263
2602399210,20cyclictest0-21swapper/308:55:413
2602399210,0cyclictest19794-21diskmemload10:30:293
26022992121,0cyclictest19794-21diskmemload09:15:112
2602299210,20cyclictest0-21swapper/207:50:412
26021992121,0cyclictest3258-21apt-get08:35:421
26021992120,1cyclictest0-21swapper/108:25:411
26021992119,2cyclictest8582-21sendmail-msp07:40:001
2602199210,21cyclictest0-21swapper/107:10:411
2602199210,20cyclictest0-21swapper/107:55:401
26020992121,0cyclictest0-21swapper/008:35:420
26020992121,0cyclictest0-21swapper/008:00:000
26020992120,1cyclictest15333-21apt-get09:00:420
2602099210,1cyclictest26314-21apt-get07:10:410
2602099210,1cyclictest13362-21cat07:50:000
26023992020,0cyclictest0-21swapper/308:50:423
26023992020,0cyclictest0-21swapper/308:45:003
26023992020,0cyclictest0-21swapper/307:55:403
26023992018,2cyclictest6062-21munin-run07:35:013
2602399200,20cyclictest0-21swapper/309:05:423
2602399200,15cyclictest18477-21apt-get08:00:403
2602399200,0cyclictest6353-21apt-get07:35:403
2602399200,0cyclictest0-21swapper/307:40:403
26022992018,1cyclictest12914-21apt-get08:55:412
2602299200,0cyclictest0-21swapper/209:00:422
26021992020,0cyclictest3905-21apt-get07:30:411
26020992020,0cyclictest0-21swapper/008:15:410
26020992020,0cyclictest0-21swapper/008:10:420
26020992018,1cyclictest496-21perf08:30:000
26020992018,1cyclictest12595-21sh08:55:010
26020992018,1cyclictest10226-21munin-run08:50:000
2602099200,0cyclictest0-21swapper/008:05:400
26023991919,0cyclictest0-21swapper/308:20:023
26023991918,0cyclictest0-21swapper/307:50:423
26022991917,1cyclictest478-21cron08:30:002
26021991918,1cyclictest3588-21perf07:30:011
26020991919,0cyclictest0-21swapper/007:50:410
26023991818,0cyclictest0-21swapper/307:15:403
26023991815,2cyclictest25356-21perf08:15:013
2602399180,0cyclictest0-21swapper/307:10:403
26022991817,1cyclictest0-21swapper/207:35:402
26021991818,0cyclictest0-21swapper/108:45:431
26020991818,0cyclictest0-21swapper/008:55:410
26020991818,0cyclictest0-21swapper/008:00:400
26020991818,0cyclictest0-21swapper/007:30:410
26020991816,1cyclictest31211-21perf07:20:010
26023991717,0cyclictest0-21swapper/308:35:423
26023991717,0cyclictest0-21swapper/307:20:423
26022991717,0cyclictest0-21swapper/208:20:432
26022991717,0cyclictest0-21swapper/207:10:412
26021991715,1cyclictest11269-21apt-get07:45:411
26023991616,0cyclictest0-21swapper/308:25:413
26023991614,1cyclictest3222-21users07:25:493
26022991616,0cyclictest0-21swapper/207:45:402
26022991615,0cyclictest31523-21apt-get07:20:422
26022991614,1cyclictest0-21swapper/208:05:412
26021991616,0cyclictest0-21swapper/108:20:421
2602199160,16cyclictest0-21swapper/107:53:331
26020991616,0cyclictest0-21swapper/008:20:020
26022991515,0cyclictest16059-21apt-get07:55:392
2602299150,1cyclictest4361-21ntp_states08:35:472
2602199150,15cyclictest0-21swapper/108:32:081
26020991515,0cyclictest0-21swapper/008:40:410
2602099150,1cyclictest0-21swapper/007:20:470
2602099150,14cyclictest0-21swapper/007:29:250
26023991414,0cyclictest0-21swapper/309:00:203
26023991414,0cyclictest0-21swapper/308:45:373
26022991414,0cyclictest0-21swapper/208:15:172
26022991414,0cyclictest0-21swapper/208:10:112
2602299140,6cyclictest0-21swapper/207:25:392
2602299140,0cyclictest0-21swapper/208:40:092
26021991414,0cyclictest0-21swapper/109:00:401
2602099140,0cyclictest0-21swapper/009:05:100
2602099140,0cyclictest0-21swapper/007:41:260
2602099140,0cyclictest0-21swapper/007:37:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional