You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 11:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Apr 30, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28006993230,1cyclictest13-21ksoftirqd/022:05:010
28012992519,5cyclictest11323-21ssh00:36:202
2801299250,24cyclictest9435-21ssh23:19:052
28007992524,1cyclictest0-21swapper/123:35:321
2800799250,24cyclictest0-21swapper/121:44:221
28006992523,1cyclictest31714-21ssh21:50:220
2800699250,24cyclictest22081-21diskmemload21:46:440
2800699250,1cyclictest141rcuc/022:46:310
28013992423,1cyclictest3132-21ssh23:12:053
2801399240,24cyclictest22081-21diskmemload22:15:283
2801399240,23cyclictest32703-21perf21:20:013
2801399240,23cyclictest22081-21diskmemload23:21:233
28012992422,1cyclictest27843-21sh21:15:032
28012992422,1cyclictest0-21swapper/221:45:552
2801299241,22cyclictest1370-21kworker/u8:122:39:572
2801299240,23cyclictest32042-21ssh21:50:522
2801299240,23cyclictest22081-21diskmemload00:21:062
2801299240,23cyclictest17792-21ssh22:12:532
2801299240,23cyclictest0-21swapper/223:03:012
2801299240,23cyclictest0-21swapper/221:13:002
28007992423,1cyclictest0-21swapper/100:06:081
28007992422,1cyclictest0-21swapper/120:05:021
2800799240,24cyclictest0-21swapper/122:50:541
2800799240,23cyclictest9968-21sh22:03:551
2800799240,23cyclictest0-21swapper/123:04:241
2800799240,23cyclictest0-21swapper/100:02:261
2800799240,22cyclictest774-21runrttasks21:33:111
28006992423,1cyclictest27939-21ssh23:40:360
28006992422,1cyclictest0-21swapper/023:55:440
28006992422,1cyclictest0-21swapper/000:24:590
2800699240,24cyclictest0-21swapper/021:57:060
2800699240,24cyclictest0-21swapper/021:34:260
2800699240,23cyclictest26728-21ssh23:01:320
2800699240,23cyclictest0-21swapper/022:27:430
2800699240,1cyclictest0-21swapper/023:06:010
28013992323,0cyclictest0-21swapper/322:25:553
28013992323,0cyclictest0-21swapper/322:11:113
28013992322,1cyclictest22081-21diskmemload21:24:253
28013992320,2cyclictest22117-21cut21:10:003
2801399230,23cyclictest0-21swapper/322:40:083
2801399230,23cyclictest0-21swapper/321:50:303
2801399230,22cyclictest2821-21rm00:26:503
2801399230,22cyclictest27327-21ssh23:02:263
2801399230,22cyclictest27186-21ssh23:39:533
2801399230,22cyclictest25610-21ssh21:44:043
2801399230,22cyclictest22081-21diskmemload21:56:183
2801399230,22cyclictest22081-21diskmemload00:37:593
2801399230,22cyclictest16337-21sh21:35:113
2801399230,22cyclictest12659-21ssh22:45:333
2801399230,22cyclictest1226-21rm23:09:303
2801399230,22cyclictest0-21swapper/321:12:313
2801399230,1cyclictest13918-21sh22:07:043
2801399230,0cyclictest22081-21diskmemload22:53:063
2801399230,0cyclictest0-21swapper/323:56:033
2801399230,0cyclictest0-21swapper/323:32:553
28012992323,0cyclictest0-21swapper/223:40:312
28012992323,0cyclictest0-21swapper/223:32:182
28012992323,0cyclictest0-21swapper/221:58:212
28012992323,0cyclictest0-21swapper/221:32:222
28012992323,0cyclictest0-21swapper/200:11:492
28012992323,0cyclictest0-21swapper/200:03:392
28012992322,1cyclictest22081-21diskmemload00:30:452
28012992322,0cyclictest22081-21diskmemload23:22:572
28012992321,1cyclictest313-21systemd-journal19:15:012
28012992321,1cyclictest19908-21apt-get22:15:552
28012992321,1cyclictest0-21swapper/223:54:102
28012992321,1cyclictest0-21swapper/222:56:482
28012992320,2cyclictest25071-21rm21:43:152
28012992320,2cyclictest0-21swapper/222:50:342
2801299230,23cyclictest0-21swapper/223:27:352
2801299230,22cyclictest9464-21ssh23:56:212
2801299230,22cyclictest3218-21sh00:27:262
2801299230,22cyclictest12891-21ssh22:06:012
2801299230,22cyclictest0-21swapper/222:31:412
2801299230,21cyclictest151rcu_preempt22:26:482
2801299230,1cyclictest2460-21munin-run20:30:002
2801299230,0cyclictest0-21swapper/221:29:562
28007992323,0cyclictest22081-21diskmemload00:22:441
28007992323,0cyclictest0-21swapper/123:05:271
28007992323,0cyclictest0-21swapper/121:58:311
28007992322,1cyclictest12424-21perf19:45:011
28007992321,1cyclictest0-21swapper/123:47:341
28007992321,1cyclictest0-21swapper/123:28:011
28007992321,1cyclictest0-21swapper/100:31:181
28007992321,1cyclictest0-21swapper/100:25:111
2800799230,23cyclictest0-21swapper/123:18:391
2800799230,23cyclictest0-21swapper/121:51:521
2800799230,23cyclictest0-21swapper/121:45:101
2800799230,22cyclictest25396-21kworker/u8:121:21:001
2800799230,22cyclictest22081-21diskmemload21:36:051
2800799230,22cyclictest22081-21diskmemload21:30:011
2800799230,22cyclictest22081-21diskmemload21:11:151
2800799230,22cyclictest0-21swapper/122:33:141
2800799230,1cyclictest0-21swapper/122:45:441
2800799230,1cyclictest0-21swapper/119:20:001
2800799230,0cyclictest0-21swapper/123:56:171
2800799230,0cyclictest0-21swapper/122:24:541
2800699233,20cyclictest62550irq/127-eno123:35:480
28006992323,0cyclictest0-21swapper/000:15:470
28006992322,1cyclictest13-21ksoftirqd/021:20:010
28006992321,1cyclictest19106-21sh23:30:340
28006992321,1cyclictest0-21swapper/022:40:440
28006992321,1cyclictest0-21swapper/020:05:020
28006992321,1cyclictest0-21swapper/000:02:050
2800699230,2cyclictest5009-21sh21:24:240
2800699230,23cyclictest22081-21diskmemload22:20:480
2800699230,23cyclictest141rcuc/021:25:310
2800699230,23cyclictest0-21swapper/023:52:570
2800699230,23cyclictest0-21swapper/022:15:530
2800699230,23cyclictest0-21swapper/000:38:340
2800699230,22cyclictest449-21rm22:30:050
2800699230,22cyclictest32753-21perf19:20:000
2800699230,22cyclictest2872-21ssh23:49:170
2800699230,22cyclictest2532-21ssh23:11:110
2800699230,22cyclictest22081-21diskmemload21:38:020
2800699230,22cyclictest0-21swapper/022:51:070
2800699230,22cyclictest0-21swapper/000:08:380
28013992222,0cyclictest0-21swapper/323:52:313
28013992222,0cyclictest0-21swapper/322:35:243
28013992219,2cyclictest14881-21sh20:55:003
2801399220,22cyclictest0-21swapper/300:23:273
2801399220,21cyclictest7903-21sh22:00:303
2801399220,21cyclictest5283-21ssh23:15:133
2801399220,21cyclictest31512-21apt-get23:45:533
2801399220,21cyclictest12179-21ssh00:00:043
2801399220,21cyclictest0-21swapper/323:25:023
2801399220,1cyclictest606-21dbus-daemon19:35:003
2801399220,1cyclictest31047-21ssh21:49:443
2801399220,1cyclictest22081-21diskmemload00:16:283
2801399220,1cyclictest0-21swapper/321:30:583
2801399220,0cyclictest0-21swapper/322:24:313
2801399220,0cyclictest0-21swapper/319:20:533
28012992222,0cyclictest0-21swapper/222:21:042
28012992222,0cyclictest0-21swapper/221:21:072
28012992220,1cyclictest12406-21apt-get19:45:022
2801299220,21cyclictest2778-21apt-get20:30:552
2801299220,21cyclictest2704-21sh23:11:272
2801299220,21cyclictest22081-21diskmemload23:36:252
2801299220,21cyclictest22081-21diskmemload00:15:052
2801299220,21cyclictest0-21swapper/223:05:352
2801299220,21cyclictest0-21swapper/222:46:172
2801299220,21cyclictest0-21swapper/200:05:052
2801299220,1cyclictest7694-21ssh22:00:112
28007992222,0cyclictest0-21swapper/123:10:061
28007992222,0cyclictest0-21swapper/122:40:061
28007992222,0cyclictest0-21swapper/100:18:451
28007992221,0cyclictest22081-21diskmemload23:31:011
28007992220,1cyclictest21812-21ssh22:16:111
2800799220,22cyclictest0-21swapper/123:20:301
2800799220,21cyclictest22081-21diskmemload00:10:101
2800799220,21cyclictest15869-21hddtemp_smartct20:55:561
2800799220,21cyclictest11392-21sh22:05:461
2800799220,21cyclictest0-21swapper/123:50:591
2800799220,21cyclictest0-21swapper/122:25:361
2800799220,21cyclictest0-21swapper/122:10:211
2800799220,21cyclictest0-21swapper/120:45:541
2800799220,21cyclictest0-21swapper/120:00:541
2800799220,21cyclictest0-21swapper/119:45:521
2800799220,21cyclictest0-21swapper/100:35:441
2800799220,1cyclictest0-21swapper/122:36:201
28006992222,0cyclictest0-21swapper/023:21:100
28006992220,1cyclictest2439-21latency_hist20:30:000
28006992220,1cyclictest23255-21ptp4l21:41:020
28006992220,1cyclictest22429-21ssh22:56:000
28006992220,1cyclictest0-21swapper/020:35:010
2800699220,21cyclictest30026-21munin-run20:20:000
2800699220,21cyclictest28282-21apt-get19:10:530
2800699220,21cyclictest22081-21diskmemload23:25:200
2800699220,21cyclictest22081-21diskmemload23:15:050
2800699220,21cyclictest22081-21diskmemload22:10:210
2800699220,21cyclictest0-21swapper/022:09:450
2800699220,1cyclictest15212-21latency_hist19:50:010
2800699220,1cyclictest0-21swapper/000:10:490
2800699220,0cyclictest0-21swapper/021:14:280
2800699220,0cyclictest0-21swapper/019:55:530
28013992121,0cyclictest0-21swapper/300:30:443
28013992120,1cyclictest22081-21diskmemload22:58:293
28013992120,1cyclictest16715-21sh00:05:563
28013992120,0cyclictest0-21swapper/300:10:133
2801399210,21cyclictest22081-21diskmemload23:42:263
2801399210,21cyclictest0-21swapper/321:26:223
2801399210,20cyclictest578-21sh22:30:163
2801399210,20cyclictest0-21swapper/320:25:523
2801399210,20cyclictest0-21swapper/320:05:023
2801399210,0cyclictest0-21swapper/320:00:543
28012992120,0cyclictest25523-21apt-get20:10:522
28012992119,1cyclictest9321-21ssh22:40:422
2801299210,21cyclictest0-21swapper/221:35:232
2801299210,21cyclictest0-21swapper/220:40:542
2801299210,20cyclictest17923-21apt-get19:55:532
2801299210,20cyclictest15222-21apt-get20:55:542
2801299210,20cyclictest0-21swapper/219:50:532
2801299210,0cyclictest20340-21apt-get20:00:542
28007992121,0cyclictest0-21swapper/121:05:541
2800799210,0cyclictest0-21swapper/121:15:381
28006992121,0cyclictest0-21swapper/020:45:020
28006992120,1cyclictest13-21ksoftirqd/019:30:000
28006992119,1cyclictest17601-21idleruntime-cro19:55:010
2800699210,20cyclictest0-21swapper/020:55:530
2800699210,0cyclictest0-21swapper/000:25:290
28013992020,0cyclictest0-21swapper/319:55:533
28013992020,0cyclictest0-21swapper/319:45:023
28013992019,1cyclictest0-21swapper/320:25:003
2801399200,0cyclictest0-21swapper/321:00:553
28012992020,0cyclictest0-21swapper/219:20:532
2801299200,0cyclictest0-21swapper/221:00:552
28007992019,1cyclictest7874-21apt-get19:35:541
28006992019,1cyclictest13-21ksoftirqd/020:35:010
2800699200,20cyclictest0-21swapper/020:40:540
28013991919,0cyclictest27922-21apt-get20:15:543
28013991919,0cyclictest0-21swapper/320:45:543
28012991919,0cyclictest0-21swapper/220:15:542
28007991919,0cyclictest0-21swapper/119:30:531
28006991919,0cyclictest0-21swapper/020:10:520
28006991917,1cyclictest13-21ksoftirqd/021:10:010
28013991818,0cyclictest0-21swapper/320:30:553
28013991818,0cyclictest0-21swapper/319:40:533
28012991818,0cyclictest0-21swapper/219:30:532
28012991816,1cyclictest10289-21apt-get19:40:532
28007991818,0cyclictest0-21swapper/120:20:541
28007991818,0cyclictest0-21swapper/120:10:521
28006991818,0cyclictest0-21swapper/020:50:540
28006991817,1cyclictest657-21apt-get19:20:530
28013991717,0cyclictest0-21swapper/320:35:543
28013991717,0cyclictest0-21swapper/319:10:533
28012991717,0cyclictest0-21swapper/220:05:552
28012991717,0cyclictest0-21swapper/219:35:542
28012991717,0cyclictest0-21swapper/219:15:522
28012991715,1cyclictest5174-21apt-get20:35:552
28006991717,0cyclictest0-21swapper/020:20:540
28013991616,0cyclictest3080-21apt-get19:25:543
2801399160,15cyclictest7590-21apt-get20:40:543
28007991616,0cyclictest0-21swapper/120:35:551
28007991616,0cyclictest0-21swapper/119:10:531
28007991615,1cyclictest4859-21munin-run20:35:011
28007991615,1cyclictest17640-21apt-get21:00:551
2800799160,0cyclictest0-21swapper/119:20:541
28006991616,0cyclictest0-21swapper/021:02:490
28006991616,0cyclictest0-21swapper/019:40:530
28006991614,1cyclictest8515-21hddtemp_smartct19:35:560
2800699160,15cyclictest0-21swapper/019:30:590
28013991514,1cyclictest0-21swapper/319:51:143
28013991513,1cyclictest32743-21awk19:20:003
28012991515,0cyclictest0-21swapper/220:50:542
28012991515,0cyclictest0-21swapper/219:25:542
28012991514,0cyclictest20031-21apt-get21:05:542
28007991515,0cyclictest0-21swapper/119:50:531
2800799150,14cyclictest0-21swapper/120:50:531
28013991414,0cyclictest0-21swapper/320:10:023
2801399140,14cyclictest0-21swapper/320:55:193
2801399140,0cyclictest0-21swapper/319:35:323
28012991414,0cyclictest0-21swapper/220:20:122
28012991412,1cyclictest9660-21apt-get20:45:022
28007991414,0cyclictest0-21swapper/119:25:241
2800799140,14cyclictest0-21swapper/119:55:171
2800799140,0cyclictest0-21swapper/120:40:291
2800699140,0cyclictest0-21swapper/020:00:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional