You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 21:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon May 11, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24528992726,1cyclictest28-21ksoftirqd/210:23:352
2452899261,1cyclictest28-21ksoftirqd/210:16:552
2452799260,25cyclictest25870-21sendmail08:20:001
2452999250,24cyclictest10970-21awk11:25:013
2452899250,24cyclictest16020-21ssh10:32:152
2452799250,24cyclictest26148-21sh11:09:531
2452799250,24cyclictest10344-21ssh10:56:291
2452799250,24cyclictest0-21swapper/112:29:041
2452799250,24cyclictest0-21swapper/111:31:401
2452799250,24cyclictest0-21swapper/110:54:001
2452799250,24cyclictest0-21swapper/109:35:391
24529992423,1cyclictest18289-21diskmemload09:25:303
24529992422,1cyclictest0-21swapper/310:56:533
2452999240,23cyclictest18289-21diskmemload12:14:193
2452999240,23cyclictest18289-21diskmemload10:31:323
2452999240,23cyclictest18289-21diskmemload09:36:173
2452999240,23cyclictest1445-21apt-get08:36:513
2452999240,23cyclictest0-21swapper/311:59:223
2452999240,1cyclictest30383-21sh09:48:193
2452899240,24cyclictest0-21swapper/212:12:052
2452899240,23cyclictest18289-21diskmemload11:05:012
2452899240,23cyclictest15630-21ssh12:26:082
2452899240,23cyclictest0-21swapper/209:28:142
2452899240,1cyclictest18289-21diskmemload11:39:112
2452899240,0cyclictest271rcuc/211:53:562
2452899240,0cyclictest0-21swapper/211:30:002
24527992424,0cyclictest0-21swapper/109:44:421
24527992422,1cyclictest774-21runrttasks10:30:421
24527992422,1cyclictest4088-21ntp_states11:46:571
24527992422,1cyclictest0-21swapper/110:16:571
2452799240,24cyclictest27167-21kworker/u8:011:16:441
2452799240,23cyclictest0-21swapper/111:56:111
2452799240,23cyclictest0-21swapper/111:54:431
2452799240,23cyclictest0-21swapper/111:42:111
2452799240,0cyclictest18289-21diskmemload09:16:321
24526992422,1cyclictest0-21swapper/010:06:450
24526992421,2cyclictest13564-21/usr/sbin/munin10:01:590
24526992421,2cyclictest0-21swapper/010:43:520
2452699240,23cyclictest18289-21diskmemload12:05:550
2452699240,23cyclictest18289-21diskmemload11:37:300
2452699240,22cyclictest24869-21sh09:43:350
2452699240,1cyclictest18289-21diskmemload09:46:060
2452699240,0cyclictest18289-21diskmemload10:12:580
2452699240,0cyclictest0-21swapper/012:20:450
2452699240,0cyclictest0-21swapper/010:15:380
2452699240,0cyclictest0-21swapper/009:25:230
24529992323,0cyclictest18289-21diskmemload11:42:283
24529992323,0cyclictest0-21swapper/311:10:123
24529992323,0cyclictest0-21swapper/309:18:413
24529992322,1cyclictest18289-21diskmemload09:34:063
24529992322,1cyclictest18289-21diskmemload09:34:063
24529992321,1cyclictest0-21swapper/310:45:513
2452999230,2cyclictest25975-21cpuspeed_turbos12:06:533
2452999230,2cyclictest11208-21ssh10:28:293
2452999230,2cyclictest10050-21sh12:21:173
2452999230,23cyclictest31817-21ssh10:17:503
2452999230,23cyclictest0-21swapper/311:17:193
2452999230,22cyclictest6610-21perf07:40:003
2452999230,22cyclictest23093-21ssh10:38:243
2452999230,22cyclictest19771-21ssh09:11:393
2452999230,22cyclictest1793-21sh10:20:463
2452999230,22cyclictest16859-21munin-run11:30:003
2452999230,0cyclictest18289-21diskmemload12:35:063
2452999230,0cyclictest18289-21diskmemload11:02:493
2452999230,0cyclictest0-21swapper/312:30:293
2452999230,0cyclictest0-21swapper/310:51:073
2452999230,0cyclictest0-21swapper/310:10:483
2452999230,0cyclictest0-21swapper/309:56:183
24528992323,0cyclictest18289-21diskmemload11:16:102
24528992323,0cyclictest0-21swapper/212:09:512
24528992323,0cyclictest0-21swapper/211:40:412
24528992323,0cyclictest0-21swapper/210:35:512
24528992323,0cyclictest0-21swapper/209:45:342
24528992323,0cyclictest0-21swapper/209:24:062
24528992322,1cyclictest18289-21diskmemload09:52:082
24528992322,1cyclictest16924-21ssh11:30:022
24528992321,1cyclictest13055-21sh11:55:032
24528992321,1cyclictest0-21swapper/209:39:182
24528992319,4cyclictest0-21swapper/209:18:462
2452899231,21cyclictest28-21ksoftirqd/209:12:122
2452899230,2cyclictest20692-21rm10:08:172
2452899230,23cyclictest0-21swapper/210:57:152
2452899230,23cyclictest0-21swapper/210:12:322
2452899230,22cyclictest18289-21diskmemload11:46:282
2452899230,22cyclictest0-21swapper/210:47:282
2452899230,22cyclictest0-21swapper/209:30:212
2452899230,22cyclictest0-21swapper/209:30:212
2452899230,0cyclictest18289-21diskmemload10:27:202
2452899230,0cyclictest0-21swapper/211:12:532
24527992323,0cyclictest31534-21apt-get09:21:521
24527992323,0cyclictest18289-21diskmemload10:28:211
24527992323,0cyclictest0-21swapper/111:23:251
24527992321,1cyclictest0-21swapper/112:20:381
24527992321,1cyclictest0-21swapper/110:23:151
24527992321,1cyclictest0-21swapper/109:46:401
2452799230,23cyclictest0-21swapper/109:56:341
2452799230,22cyclictest4400-21sh09:26:261
2452799230,22cyclictest18289-21diskmemload12:35:321
2452799230,22cyclictest1-21systemd07:55:011
2452799230,22cyclictest0-21swapper/110:48:511
2452799230,22cyclictest0-21swapper/110:05:271
2452799230,22cyclictest0-21swapper/110:02:011
2452799230,1cyclictest18289-21diskmemload11:36:191
2452799230,1cyclictest0-21swapper/110:10:351
2452799230,0cyclictest0-21swapper/112:18:201
2452799230,0cyclictest0-21swapper/109:33:331
2452799230,0cyclictest0-21swapper/109:33:331
24526992323,0cyclictest18289-21diskmemload11:58:000
24526992322,1cyclictest0-21swapper/010:50:360
24526992321,1cyclictest3381-21rm11:17:310
24526992321,1cyclictest18289-21diskmemload09:10:440
24526992320,2cyclictest18766-21ssh09:38:100
24526992320,2cyclictest0-21swapper/012:19:580
2452699230,22cyclictest5878-21dump-pmu-power08:45:010
2452699230,22cyclictest19495-21ssh11:31:590
2452699230,22cyclictest18289-21diskmemload11:00:020
2452699230,22cyclictest11311-21sh11:53:020
2452699230,22cyclictest0-21swapper/010:46:520
2452699230,1cyclictest0-21swapper/010:20:060
2452699230,0cyclictest0-21swapper/011:10:490
2452699230,0cyclictest0-21swapper/010:36:230
2452699230,0cyclictest0-21swapper/009:31:230
2452699230,0cyclictest0-21swapper/009:31:230
2452699230,0cyclictest0-21swapper/009:21:100
2452699230,0cyclictest0-21swapper/009:15:580
24529992222,0cyclictest0-21swapper/310:05:113
24529992222,0cyclictest0-21swapper/309:50:103
24529992221,1cyclictest0-21swapper/311:30:473
24529992220,2cyclictest7858-21ssh11:50:463
24529992220,1cyclictest23086-21ssh11:35:393
2452999220,21cyclictest5607-21sh11:47:593
2452999220,21cyclictest23871-21users11:07:023
2452999220,21cyclictest15244-21ssh12:25:393
2452999220,21cyclictest0-21swapper/312:00:363
2452999220,21cyclictest0-21swapper/308:11:493
2452999220,1cyclictest24910-21runrttasks10:40:303
2452999220,1cyclictest12272-21ssh10:01:503
24528992222,0cyclictest0-21swapper/212:15:592
24528992222,0cyclictest0-21swapper/210:50:212
24528992221,1cyclictest28-21ksoftirqd/211:20:552
24528992221,1cyclictest271rcuc/212:35:412
24528992220,1cyclictest0-21swapper/211:00:162
2452899220,22cyclictest0-21swapper/210:40:152
2452899220,21cyclictest28282-21apt-get08:25:022
2452899220,21cyclictest271rcuc/210:00:182
2452899220,21cyclictest21299-21ssh12:31:062
2452899220,21cyclictest0-21swapper/207:16:522
2452899220,1cyclictest0-21swapper/209:55:172
24527992221,1cyclictest18289-21diskmemload12:33:271
24527992220,1cyclictest0-21swapper/111:10:081
24527992220,1cyclictest0-21swapper/110:40:071
2452799220,22cyclictest0-21swapper/111:25:041
2452799220,21cyclictest31725-21perf07:25:011
2452799220,21cyclictest18289-21diskmemload12:10:321
2452799220,21cyclictest0-21swapper/109:50:471
2452799220,1cyclictest24912-21apt-get07:11:511
2452799220,1cyclictest15557-21sh11:00:431
2452799220,1cyclictest0-21swapper/110:37:391
2452799220,1cyclictest0-21swapper/109:10:041
2452799220,0cyclictest0-21swapper/108:25:021
24526992222,0cyclictest62550irq/127-eno111:26:250
24526992222,0cyclictest2103-21apt-get07:31:500
24526992222,0cyclictest0-21swapper/012:25:440
24526992222,0cyclictest0-21swapper/012:00:390
24526992222,0cyclictest0-21swapper/011:05:080
24526992222,0cyclictest0-21swapper/010:26:460
24526992221,1cyclictest1813-21ssh11:45:170
24526992221,1cyclictest0-21swapper/011:42:220
24526992220,1cyclictest0-21swapper/009:50:520
24526992220,1cyclictest0-21swapper/008:25:020
2452699220,22cyclictest31243-21ssh12:11:350
2452699220,22cyclictest18289-21diskmemload10:55:190
2452699220,21cyclictest12757-21diskmemload10:30:160
2452699220,21cyclictest11058-21apt-get08:56:510
2452699220,21cyclictest0-21swapper/012:35:130
2452699220,21cyclictest0-21swapper/008:06:510
2452699220,1cyclictest21408-21apt-get08:11:490
2452699220,1cyclictest0-21swapper/012:30:100
2452699220,1cyclictest0-21swapper/011:23:590
2452699220,1cyclictest0-21swapper/007:46:510
2452699220,0cyclictest0-21swapper/009:56:220
24529992121,0cyclictest0-21swapper/309:21:523
24529992120,0cyclictest0-21swapper/307:31:503
24529992119,1cyclictest8252-21cron08:50:013
2452999210,21cyclictest0-21swapper/307:16:523
24528992118,2cyclictest0-21swapper/208:21:512
2452899210,20cyclictest0-21swapper/208:06:522
2452899210,20cyclictest0-21swapper/207:31:502
24527992119,1cyclictest1698-21perf07:30:001
2452799210,1cyclictest0-21swapper/109:06:521
2452799210,1cyclictest0-21swapper/108:21:511
2452799210,0cyclictest0-21swapper/108:51:521
24526992119,1cyclictest16200-21sh08:00:010
24526992119,1cyclictest16200-21sh08:00:010
24526992119,1cyclictest0-21swapper/009:05:000
2452699210,21cyclictest0-21swapper/008:46:500
2452699210,20cyclictest0-21swapper/008:31:510
2452699210,20cyclictest0-21swapper/008:01:510
2452699210,0cyclictest0-21swapper/007:11:510
24529992020,0cyclictest0-21swapper/308:56:513
24529992020,0cyclictest0-21swapper/307:56:513
24529992020,0cyclictest0-21swapper/307:56:513
24529992019,1cyclictest13782-21apt-config09:00:493
2452999200,0cyclictest9416-21apt-get07:46:513
24528992020,0cyclictest0-21swapper/208:56:512
24528992020,0cyclictest0-21swapper/208:41:512
24528992020,0cyclictest0-21swapper/208:35:002
2452899200,0cyclictest0-21swapper/208:01:522
2452899200,0cyclictest0-21swapper/207:51:512
24527992020,0cyclictest0-21swapper/108:41:521
24527992020,0cyclictest0-21swapper/107:41:521
24527992020,0cyclictest0-21swapper/107:36:521
2452799200,20cyclictest0-21swapper/108:11:491
2452799200,0cyclictest0-21swapper/107:46:511
24526992020,0cyclictest8664-21apt-get08:51:520
24526992020,0cyclictest0-21swapper/008:16:510
24526992020,0cyclictest0-21swapper/007:41:510
24526992020,0cyclictest0-21swapper/007:26:510
24526992016,3cyclictest4603-21apt-get07:36:520
24529991919,0cyclictest31472-21apt-get08:31:503
24529991919,0cyclictest23803-21apt-get08:16:513
24529991919,0cyclictest11808-21apt-get07:51:513
24529991919,0cyclictest0-21swapper/307:26:513
24529991919,0cyclictest0-21swapper/307:21:513
24528991919,0cyclictest0-21swapper/209:06:522
24528991919,0cyclictest0-21swapper/207:56:502
24528991919,0cyclictest0-21swapper/207:56:502
24528991919,0cyclictest0-21swapper/207:41:512
24527991919,0cyclictest0-21swapper/109:01:521
24527991919,0cyclictest0-21swapper/108:46:501
24527991919,0cyclictest0-21swapper/108:36:521
2452699190,0cyclictest0-21swapper/008:36:510
24528991818,0cyclictest0-21swapper/208:11:492
24528991816,1cyclictest10677-21munin-run08:55:012
2452899180,17cyclictest0-21swapper/207:36:522
24526991818,0cyclictest0-21swapper/007:16:520
24529991717,0cyclictest0-21swapper/308:51:523
24529991717,0cyclictest0-21swapper/308:01:523
24529991715,1cyclictest26286-21apt-get08:21:523
24528991717,0cyclictest0-21swapper/207:21:512
2452899170,15cyclictest3469-21sendmail-msp08:40:002
24527991715,1cyclictest13070-21perf09:00:011
24526991717,0cyclictest0-21swapper/009:06:520
24526991717,0cyclictest0-21swapper/007:21:500
24529991616,0cyclictest0-21swapper/308:26:523
24529991615,1cyclictest7023-21apt-get07:41:513
24527991614,1cyclictest0-21swapper/107:33:521
24526991614,1cyclictest0-21swapper/007:55:010
2452699160,15cyclictest0-21swapper/008:21:530
24528991514,1cyclictest6268-21apt-get08:46:502
2452899150,14cyclictest0-21swapper/208:16:502
24527991515,0cyclictest0-21swapper/107:56:511
24527991515,0cyclictest0-21swapper/107:56:511
2452799150,15cyclictest0-21swapper/108:31:591
2452799150,14cyclictest0-21swapper/108:01:541
24529991414,0cyclictest19016-21apt-get08:06:513
24529991414,0cyclictest0-21swapper/307:12:153
2452999140,0cyclictest0-21swapper/309:05:403
2452999140,0cyclictest0-21swapper/308:40:593
24528991414,0cyclictest0-21swapper/207:46:542
2452899140,14cyclictest0-21swapper/209:01:422
2452899140,14cyclictest0-21swapper/207:12:092
2452899140,0cyclictest0-21swapper/207:25:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional