You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 23:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Wed Apr 01, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
840299274,22cyclictest34-21ksoftirqd/308:40:003
8401992620,0cyclictest0-21swapper/212:20:302
8401992523,1cyclictest0-21swapper/207:55:002
8401992521,0cyclictest0-21swapper/209:56:012
840199250,24cyclictest28897-21apt-get10:08:292
8399992523,1cyclictest0-21swapper/010:50:270
8402992422,1cyclictest3075-21apt-get08:08:303
8402992422,1cyclictest3075-21apt-get08:08:303
840299240,23cyclictest2568-21diskmemload10:20:113
840299240,23cyclictest12159-21apt-get11:58:313
840299240,23cyclictest0-21swapper/311:13:313
8401992423,1cyclictest2568-21diskmemload10:15:162
8401992423,1cyclictest0-21swapper/212:27:012
8401992422,1cyclictest3351-21apt-get09:43:302
8401992422,1cyclictest2568-21diskmemload09:55:012
8401992422,1cyclictest0-21swapper/211:42:512
840199240,23cyclictest2568-21diskmemload11:12:292
840199240,23cyclictest0-21swapper/211:52:032
840199240,23cyclictest0-21swapper/210:43:152
840199240,1cyclictest13866-21apt-get08:28:302
8400992424,0cyclictest0-21swapper/110:53:031
8400992421,2cyclictest28643-21sh10:08:091
840099240,2cyclictest9640-21ssh09:48:371
840099240,23cyclictest8173-21munin-run12:25:011
840099240,23cyclictest7886-21munin-run09:15:011
840099240,23cyclictest775-21ssh11:46:371
8399992423,1cyclictest0-21swapper/009:12:450
8399992422,1cyclictest62550irq/127-eno111:56:370
8399992422,1cyclictest31592-21ls10:10:000
8399992421,2cyclictest20497-21ssh12:05:340
839999240,23cyclictest0-21swapper/009:29:460
8402992323,0cyclictest2568-21diskmemload10:41:083
8402992323,0cyclictest0-21swapper/309:31:023
8402992321,1cyclictest0-21swapper/312:16:063
8402992321,1cyclictest0-21swapper/311:05:053
8402992321,1cyclictest0-21swapper/310:27:073
8402992321,1cyclictest0-21swapper/309:53:333
840299232,0cyclictest34-21ksoftirqd/311:46:103
8402992320,2cyclictest8054-21awk10:50:013
8402992320,2cyclictest6758-21ssh09:45:473
840299230,2cyclictest20041-21munin-run12:05:013
840299230,23cyclictest0-21swapper/312:31:323
840299230,23cyclictest0-21swapper/311:52:223
840299230,23cyclictest0-21swapper/311:28:543
840299230,22cyclictest2568-21diskmemload11:02:343
840299230,22cyclictest18981-21ssh09:26:193
840299230,22cyclictest16121-21apt-get07:28:313
840299230,22cyclictest15972-21sh09:23:333
840299230,0cyclictest0-21swapper/312:28:383
840299230,0cyclictest0-21swapper/310:38:563
8401992323,0cyclictest32321-21ssh12:18:042
8401992323,0cyclictest2568-21diskmemload12:05:502
8401992323,0cyclictest0-21swapper/211:47:362
8401992323,0cyclictest0-21swapper/210:57:462
8401992323,0cyclictest0-21swapper/209:47:122
8401992323,0cyclictest0-21swapper/209:34:532
8401992322,0cyclictest0-21swapper/210:47:012
8401992321,1cyclictest0-21swapper/211:05:152
8401992321,1cyclictest0-21swapper/210:36:402
8401992320,2cyclictest20744-21ssh11:03:292
840199230,23cyclictest0-21swapper/210:32:112
840199230,23cyclictest0-21swapper/209:13:102
840199230,22cyclictest13286-21diskmemload12:30:032
840199230,22cyclictest0-21swapper/209:38:322
840199230,0cyclictest2568-21diskmemload10:00:092
8400992323,0cyclictest0-21swapper/111:51:381
8400992323,0cyclictest0-21swapper/111:07:551
8400992323,0cyclictest0-21swapper/111:00:461
8400992322,1cyclictest2568-21diskmemload10:55:031
8400992321,2cyclictest13312-21ssh09:20:181
8400992321,1cyclictest21329-21apt10:00:011
8400992320,2cyclictest26950-21ssh12:12:511
8400992320,2cyclictest1-21systemd07:55:001
840099230,2cyclictest3262-21ssh10:45:081
840099230,2cyclictest2744-21ssh10:44:451
840099230,2cyclictest26518-21turbostat.cron10:05:011
840099230,23cyclictest0-21swapper/112:15:251
840099230,23cyclictest0-21swapper/112:09:421
840099230,23cyclictest0-21swapper/111:31:091
840099230,23cyclictest0-21swapper/110:18:411
840099230,23cyclictest0-21swapper/109:35:471
840099230,22cyclictest2568-21diskmemload12:35:301
840099230,22cyclictest25345-21latency_hist08:50:001
840099230,22cyclictest23694-21ssh11:37:551
840099230,22cyclictest10350-21sh11:55:351
840099230,22cyclictest0-21swapper/110:20:461
840099230,0cyclictest2568-21diskmemload09:19:391
840099230,0cyclictest0-21swapper/111:14:271
8399992323,0cyclictest2568-21diskmemload10:40:010
8399992323,0cyclictest2568-21diskmemload09:52:110
8399992323,0cyclictest0-21swapper/011:11:150
8399992323,0cyclictest0-21swapper/010:46:160
8399992323,0cyclictest0-21swapper/009:17:230
8399992321,1cyclictest6420-21sh09:45:170
8399992321,1cyclictest606-21dbus-daemon11:05:000
8399992321,1cyclictest0-21swapper/012:35:520
839999231,21cyclictest0-21swapper/009:31:360
839999230,2cyclictest28055-21ssh11:09:150
839999230,2cyclictest25624-21ssh10:03:430
839999230,2cyclictest22922-21cron08:45:000
839999230,23cyclictest2568-21diskmemload11:49:080
839999230,23cyclictest19106-21ssh10:29:100
839999230,23cyclictest0-21swapper/012:13:140
839999230,23cyclictest0-21swapper/011:23:270
839999230,23cyclictest0-21swapper/011:15:380
839999230,22cyclictest4992-21sh10:15:460
839999230,22cyclictest27887-21ssh11:41:280
839999230,22cyclictest24509-21ssh10:34:040
839999230,22cyclictest15280-21sh12:00:180
839999230,22cyclictest13199-21ssh11:27:240
839999230,22cyclictest0-21swapper/009:58:310
839999230,1cyclictest2568-21diskmemload12:15:150
8402992222,0cyclictest2568-21diskmemload12:21:173
8402992222,0cyclictest0-21swapper/311:16:573
8402992222,0cyclictest0-21swapper/310:16:593
8402992222,0cyclictest0-21swapper/309:15:593
8402992222,0cyclictest0-21swapper/308:33:283
8402992221,1cyclictest8415-21ssh10:50:123
8402992221,1cyclictest0-21swapper/310:10:313
8402992220,1cyclictest1469-21sh09:40:293
8402992220,1cyclictest0-21swapper/312:10:393
8402992220,1cyclictest0-21swapper/311:41:093
8402992220,1cyclictest0-21swapper/310:00:283
8402992219,2cyclictest26851-21sh10:05:113
840299220,21cyclictest3940-21ssh09:11:463
840299220,21cyclictest2568-21diskmemload11:36:103
840299220,21cyclictest0-21swapper/309:03:303
840299220,20cyclictest151rcu_preempt09:35:213
840299220,1cyclictest331rcuc/312:35:023
840299220,1cyclictest0-21swapper/311:20:273
840299220,1cyclictest0-21swapper/310:58:393
840299220,0cyclictest0-21swapper/312:05:303
8401992222,0cyclictest2568-21diskmemload10:10:352
8401992222,0cyclictest2568-21diskmemload09:25:382
8401992222,0cyclictest0-21swapper/210:50:152
8401992221,1cyclictest0-21swapper/211:25:592
8401992221,1cyclictest0-21swapper/209:20:302
8401992220,1cyclictest2584-21sh09:10:012
8401992220,1cyclictest0-21swapper/210:21:062
8401992219,3cyclictest0-21swapper/211:22:222
840199220,21cyclictest25319-21perf07:45:002
840199220,21cyclictest13726-21apt-get07:23:302
840199220,21cyclictest0-21swapper/211:15:162
840199220,21cyclictest0-21swapper/210:26:412
840199220,1cyclictest26563-21ssh12:12:062
840199220,1cyclictest18660-21apt-get08:38:302
840199220,1cyclictest0-21swapper/212:38:142
840199220,1cyclictest0-21swapper/211:35:052
840199220,1cyclictest0-21swapper/211:30:382
840199220,0cyclictest0-21swapper/211:55:362
8400992222,0cyclictest0-21swapper/108:28:301
8400992221,1cyclictest2735-21ssh11:17:061
8400992220,1cyclictest15632-21cstates12:33:301
8400992220,1cyclictest0-21swapper/111:20:091
840099220,22cyclictest0-21swapper/112:27:441
840099220,22cyclictest0-21swapper/112:01:571
840099220,21cyclictest5865-21apt08:13:311
840099220,21cyclictest19614-21perf10:30:001
840099220,21cyclictest18293-21ssh09:25:091
840099220,21cyclictest12961-21idleruntime-cro08:25:011
840099220,21cyclictest0-21swapper/111:25:331
840099220,21cyclictest0-21swapper/109:40:321
840099220,21cyclictest0-21swapper/107:18:301
840099220,1cyclictest11724-21ssh09:50:361
840099220,0cyclictest0-21swapper/111:44:251
8399992222,0cyclictest2568-21diskmemload12:27:010
8399992222,0cyclictest2568-21diskmemload09:20:080
8399992222,0cyclictest0-21swapper/010:35:130
8399992222,0cyclictest0-21swapper/010:10:230
8399992221,1cyclictest26124-21ssh11:38:540
8399992221,1cyclictest2568-21diskmemload11:51:360
8399992221,1cyclictest2568-21diskmemload11:30:550
8399992220,2cyclictest10039-21sh10:20:390
839999220,22cyclictest0-21swapper/009:37:270
839999220,21cyclictest13460-21rm12:30:180
839999220,21cyclictest0-21swapper/008:13:310
839999220,0cyclictest0-21swapper/012:23:040
8402992119,1cyclictest4951-21apt-get08:10:023
840299210,21cyclictest0-21swapper/309:55:183
840299210,20cyclictest0-21swapper/307:53:303
840299210,20cyclictest0-21swapper/307:33:283
840199210,21cyclictest0-21swapper/209:15:172
840199210,21cyclictest0-21swapper/208:10:022
840199210,20cyclictest0-21swapper/208:23:312
840199210,20cyclictest0-21swapper/207:48:302
840199210,0cyclictest0-21swapper/212:01:082
8400992121,0cyclictest0-21swapper/108:18:311
8400992119,1cyclictest25285-21idleruntime-cro07:45:001
8400992119,1cyclictest20537-21turbostat08:40:001
840099210,21cyclictest0-21swapper/110:35:451
840099210,21cyclictest0-21swapper/110:10:141
840099210,21cyclictest0-21swapper/109:08:301
840099210,0cyclictest0-21swapper/108:53:301
8399992121,0cyclictest0-21swapper/008:33:280
839999210,21cyclictest0-21swapper/010:57:250
839999210,20cyclictest28228-21apt-get07:53:300
839999210,0cyclictest0-21swapper/009:40:580
8402992019,1cyclictest30673-21apt-get07:58:313
8402992018,1cyclictest34-21ksoftirqd/307:50:003
8401992020,0cyclictest0-21swapper/207:28:312
8401992020,0cyclictest0-21swapper/207:18:302
840199200,0cyclictest0-21swapper/208:08:302
840199200,0cyclictest0-21swapper/208:08:302
8400992020,0cyclictest0-21swapper/107:48:301
840099200,19cyclictest0-21swapper/108:08:301
840099200,19cyclictest0-21swapper/108:08:301
840099200,0cyclictest8912-21apt-get07:13:301
840099200,0cyclictest0-21swapper/107:38:291
8399992020,0cyclictest0-21swapper/007:13:300
839999200,0cyclictest0-21swapper/008:18:310
839999200,0cyclictest0-21swapper/007:48:300
8402991919,0cyclictest0-21swapper/308:48:313
8402991916,2cyclictest34-21ksoftirqd/309:00:013
840299190,0cyclictest0-21swapper/309:08:303
8401991919,0cyclictest0-21swapper/209:03:302
8401991919,0cyclictest0-21swapper/208:48:312
8401991917,1cyclictest22956-21idleruntime-cro08:45:002
840199190,0cyclictest0-21swapper/207:13:302
8400991919,0cyclictest0-21swapper/107:23:301
8399991919,0cyclictest0-21swapper/008:03:290
8399991919,0cyclictest0-21swapper/007:43:290
8402991816,1cyclictest22918-21cron08:45:003
8401991818,0cyclictest0-21swapper/208:03:292
8401991816,1cyclictest0-21swapper/208:55:012
8400991818,0cyclictest0-21swapper/107:33:281
8399991816,1cyclictest15745-21perf08:30:000
8399991816,1cyclictest11083-21apt-get08:23:310
8402991717,0cyclictest0-21swapper/307:18:303
8402991715,1cyclictest15755-21basename08:30:003
8402991713,3cyclictest10174-21sendmail-msp08:20:013
8400991717,0cyclictest0-21swapper/107:28:311
8399991717,0cyclictest0-21swapper/007:28:310
8399991715,1cyclictest13195-21perf07:20:010
839999170,16cyclictest681-21apt-get09:08:290
8402991614,1cyclictest646-21apt-get08:03:293
8402991614,1cyclictest25279-21cron07:45:003
840299160,1cyclictest12923-21cron08:25:013
8401991616,0cyclictest0-21swapper/207:38:292
8400991614,1cyclictest22951-21turbostat08:45:001
8399991614,1cyclictest0-21swapper/008:38:300
840299150,1cyclictest34-21ksoftirqd/307:38:563
8401991514,0cyclictest28259-21apt-get08:58:292
8400991515,0cyclictest30730-21apt-get09:03:301
8399991515,0cyclictest20909-21apt-get07:38:290
8399991515,0cyclictest0-21swapper/008:58:290
839999150,15cyclictest24016-21tune2fs08:48:330
840299140,14cyclictest0-21swapper/307:20:093
840299140,14cyclictest0-21swapper/307:11:323
840299140,0cyclictest0-21swapper/308:50:403
8401991414,0cyclictest0-21swapper/208:30:032
840199140,14cyclictest0-21swapper/207:55:482
840199140,0cyclictest0-21swapper/208:15:202
840199140,0cyclictest0-21swapper/207:30:562
8400991414,0cyclictest0-21swapper/108:55:161
840099140,14cyclictest0-21swapper/108:30:021
840099140,14cyclictest0-21swapper/108:01:121
839999140,14cyclictest0-21swapper/008:50:420
839999140,14cyclictest0-21swapper/007:56:080
839999140,0cyclictest0-21swapper/009:00:080
839999140,0cyclictest0-21swapper/008:05:240
839999140,0cyclictest0-21swapper/008:05:240
839999140,0cyclictest0-21swapper/007:20:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional