You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 13:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Aug 12, 2026 00:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4100002991591,132cyclictest0-21swapper/021:00:010
4100011997918,42cyclictest364-21snmpd23:20:022
4100011997918,42cyclictest364-21snmpd23:20:022
4100002997730,23cyclictest16950irq/224-800000022:55:030
4100002997730,23cyclictest16950irq/224-800000022:55:030
4100007997130,36cyclictest309-21in:imuxsock22:55:011
4100016997028,38cyclictest4188163-21sh00:02:203
4100002997029,22cyclictest0-21swapper/023:08:590
410001699692,28cyclictest4179777-21sshd23:44:563
410000799691,18cyclictest411138750ssh21:20:301
410001699683,59cyclictest4103825-21sshd20:22:443
4100007996825,36cyclictest1-21systemd20:40:011
4100007996825,36cyclictest1-21systemd20:40:001
410001699672,60cyclictest364-21snmpd00:00:013
4100016996634,29cyclictest275-21dbus-daemon22:35:013
410001699662,60cyclictest4107993-21ssh21:13:483
410000299661,47cyclictest4160897-21ssh23:05:000
4100016996523,37cyclictest4106219-21diskmemload21:23:173
4100016996519,42cyclictest4128012-1kworker/u9:022:44:183
410001699651,20cyclictest4125182-21/usr/sbin/munin21:50:023
4100007996521,39cyclictest309-21in:imuxsock21:35:001
4100002996521,40cyclictest12-21ksoftirqd/019:55:020
410001699641,4cyclictest4162903-21ssh23:09:143
410001699641,10cyclictest3467602-21systemd-journal23:20:403
410001699632,55cyclictest4144225-21cron22:30:003
410001699631,5cyclictest357-21ssh00:15:083
410001699631,12cyclictest4167482-21ssh23:18:493
410001699631,12cyclictest4167482-21ssh23:18:493
4100007996323,36cyclictest23-21ksoftirqd/122:11:261
4100007996322,35cyclictest4106673-21sshd21:10:441
410001699622,14cyclictest4135844-21ssh22:12:163
410001699622,12cyclictest4193457-21ssh00:13:363
410001699621,9cyclictest10597-21ssh00:36:473
410001699621,4cyclictest4743-21ssh00:24:533
410001699621,4cyclictest4182550-21ssh23:50:183
410001699621,4cyclictest4129773-21ssh21:59:423
410001699621,13cyclictest4140973-21ssh22:22:593
410000799623,17cyclictest4115650-21ssh21:30:011
4100002996214,8cyclictest0-21swapper/023:43:440
4100016996127,28cyclictest275-21dbus-daemon20:10:013
410001699612,6cyclictest6256-21ssh00:27:513
410001699612,5cyclictest4115655-21munin-run21:30:013
410001699612,55cyclictest4132362-21munin-run22:05:013
410001699612,53cyclictest4172590-21ssh23:29:413
410001699612,20cyclictest4134587-21ssh22:09:513
410001699611,8cyclictest4181934-21ssh23:49:293
410001699611,8cyclictest4160492-21ssh23:04:123
410001699611,8cyclictest4157716-21ssh22:58:163
410001699611,8cyclictest4157716-21ssh22:58:163
410001699611,8cyclictest4148088-21ssh22:37:533
410001699611,8cyclictest4122755-21ssh21:44:493
410001699611,6cyclictest4119133-21ssh21:36:593
410001699611,3cyclictest4174763-21ssh23:34:183
410001699611,13cyclictest8743-21ssh00:33:053
4100011996149,6cyclictest4118078-21kworker/2:022:29:132
4100002996126,10cyclictest12-21ksoftirqd/022:50:130
410001699602,5cyclictest4153763-21munin-run22:50:003
410001699602,5cyclictest4100181-21sshd19:14:123
410001699601,7cyclictest4110660-21ssh21:19:293
410001699601,55cyclictest4155006-21ssh22:52:273
410001699601,4cyclictest4163910-21ssh23:11:013
410001699601,17cyclictest4137586-21sshd22:15:473
410001699601,13cyclictest4118165-21latency_hist21:35:023
4100007996023,33cyclictest4180232-21sh23:45:251
410000799602,21cyclictest4101124-21latency_hist19:30:021
4100007996020,11cyclictest4175044-21grep23:35:011
4100002996025,30cyclictest4106219-21diskmemload22:00:030
4100002996019,32cyclictest16850irq/134-800000021:46:240
410001699592,9cyclictest4176666-21ssh23:38:153
410001699592,9cyclictest4176666-21ssh23:38:153
410001699592,6cyclictest4100420-21sshd19:16:123
410001699592,6cyclictest4100420-21sshd19:16:123
410001699591,7cyclictest4125791-21ssh21:50:523
410001699591,5cyclictest4191104-21ssh00:08:423
4100011995936,16cyclictest1-21systemd22:40:012
4100011995934,17cyclictest4186999-21tr00:00:002
4100007995920,12cyclictest23-21ksoftirqd/100:04:401
4100007995919,5cyclictest4171361-21sh23:26:481
4100002995950,6cyclictest12-21ksoftirqd/021:39:440
4100002995917,37cyclictest16850irq/134-800000021:54:040
4100002995914,10cyclictest12-21ksoftirqd/021:20:010
410001699582,7cyclictest4105405-21sshd20:50:453
410001699582,12cyclictest4100753-21munin-run19:25:013
4100011995835,15cyclictest3467602-21systemd-journal23:10:002
410000799585,17cyclictest4110929-21munin-run21:20:001
4100007995844,11cyclictest23-21ksoftirqd/121:45:021
410000799582,16cyclictest4192872-21ssh00:12:121
410000799582,14cyclictest3467602-21systemd-journal21:40:011
4100007995817,4cyclictest4101227-21cron19:35:011
4100007995814,4cyclictest0-21swapper/121:45:551
410000799581,17cyclictest4184581-21cron23:55:001
410000299582,23cyclictest4144125-21ssh22:29:470
410001699573,4cyclictest364-21snmpd20:26:243
410001699571,5cyclictest4101208-21sshd19:30:323
410001699571,10cyclictest4103529-21sshd20:15:443
410000799574,16cyclictest4146658-21modprobe22:35:001
4100007995724,29cyclictest23-21ksoftirqd/122:25:011
4100007995723,30cyclictest23-21ksoftirqd/121:52:061
4100007995720,33cyclictest4187261-21kworker/u8:400:35:031
4100007995719,4cyclictest4106219-21diskmemload22:43:151
4100007995717,8cyclictest4163441-21sed23:10:021
4100002995728,13cyclictest4169047-21ssh23:21:540
4100002995720,32cyclictest16850irq/134-800000000:17:200
410001699563,48cyclictest4104948-21munin-run20:45:013
410001699562,5cyclictest364-21snmpd19:26:123
410001699562,16cyclictest4105147-21modprobe20:45:013
410001699561,5cyclictest4106198-21sshd21:09:443
410001699561,12cyclictest4102808-21munin-run20:05:003
4100007995626,25cyclictest4144341-21latency_hist22:30:001
4100007995625,27cyclictest23-21ksoftirqd/122:56:091
4100007995625,27cyclictest23-21ksoftirqd/122:56:091
4100007995621,4cyclictest4106219-21diskmemload23:00:161
410000799561,17cyclictest4168125-21sed23:20:021
410000799561,17cyclictest4168125-21sed23:20:011
4100007995611,12cyclictest4177623-21sort23:40:021
4100007995611,12cyclictest4177623-21sort23:40:021
4100002995629,12cyclictest4121549-21ssh21:41:590
4100002995628,13cyclictest4137925-21ssh22:16:330
4100002995628,12cyclictest4185104-21ssh23:55:480
4100002995628,12cyclictest4134823-21ssh22:10:010
4100002995618,29cyclictest16850irq/134-800000021:22:470
4100002995616,24cyclictest0-21swapper/000:24:350
4100002995615,9cyclictest12-21ksoftirqd/020:20:010
4100002995613,11cyclictest12-21ksoftirqd/021:28:390
4100002995611,8cyclictest12-21ksoftirqd/020:15:010
410001699552,5cyclictest364-21snmpd20:37:483
410001699552,5cyclictest364-21snmpd20:37:483
410001699551,15cyclictest364-21snmpd21:02:273
4100011995547,5cyclictest4074770-21kworker/2:219:50:022
4100011995546,5cyclictest4153796-21kworker/2:323:40:032
4100011995546,5cyclictest4153796-21kworker/2:323:40:032
410000799559,11cyclictest0-21swapper/100:05:041
4100007995525,26cyclictest23-21ksoftirqd/123:45:011
410000799552,16cyclictest4104390-21cron20:35:001
4100007995521,5cyclictest4131268-21sh22:02:291
4100007995513,8cyclictest4105947-21sshd21:03:451
4100002995519,27cyclictest16850irq/134-800000022:48:410
4100002995516,8cyclictest12-21ksoftirqd/021:10:010
4100002995514,9cyclictest0-21swapper/023:51:400
4100002995510,10cyclictest12-21ksoftirqd/023:37:220
4100002995510,10cyclictest12-21ksoftirqd/023:37:210
4100016995427,23cyclictest37-21ksoftirqd/321:00:003
410001699541,7cyclictest4103274-21sshd20:10:363
410001699541,14cyclictest364-21snmpd19:48:463
4100011995445,5cyclictest4118078-21kworker/2:021:39:492
4100007995423,8cyclictest23-21ksoftirqd/123:55:441
4100007995413,9cyclictest4104188-21latency_hist20:30:011
410000299549,14cyclictest4141252-21sh22:23:420
4100002995429,9cyclictest4150699-21ssh22:43:280
410000299542,47cyclictest4148956-21modprobe22:40:000
4100002995418,32cyclictest16850irq/134-800000023:32:300
4100002995416,33cyclictest151rcuc/000:05:000
410000299541,37cyclictest4145145-21ssh22:31:330
410001699532,8cyclictest364-21snmpd19:36:313
410001699531,5cyclictest1-21systemd19:55:003
4100011995344,6cyclictest4074770-21kworker/2:219:20:032
4100011995344,6cyclictest4074770-21kworker/2:219:20:032
4100011995344,5cyclictest4074770-21kworker/2:219:10:032
4100011995344,5cyclictest4074770-21kworker/2:219:10:032
4100011995343,6cyclictest4191684-21kworker/2:200:24:402
410000799532,19cyclictest4134712-21latency_hist22:10:021
410000799532,15cyclictest4165673-21latency_hist23:15:011
4100007995320,4cyclictest4106219-21diskmemload21:55:411
4100007995319,12cyclictest4103298-21cron20:15:011
4100007995316,6cyclictest23-21ksoftirqd/123:22:411
4100007995316,10cyclictest23-21ksoftirqd/100:19:271
4100002995327,11cyclictest4164357-21ssh23:12:060
4100002995325,13cyclictest4191915-21ssh00:10:040
4100002995325,12cyclictest9924-21ssh00:35:130
4100002995320,25cyclictest16850irq/134-800000023:46:310
4100002995317,31cyclictest16850irq/134-800000022:01:100
4100002995313,35cyclictest0-21swapper/021:11:000
4100002995310,13cyclictest4104600-21cat20:35:020
410001699522,4cyclictest0-21swapper/320:34:213
4100011995243,6cyclictest4165625-21kworker/2:000:03:402
4100011995242,7cyclictest4153796-21kworker/2:323:54:432
4100011995242,7cyclictest4153796-21kworker/2:323:05:002
4100011995241,7cyclictest4191684-21kworker/2:200:25:132
4100011995233,13cyclictest3467602-21systemd-journal20:45:022
4100007995223,25cyclictest23-21ksoftirqd/100:27:531
4100007995221,7cyclictest23-21ksoftirqd/122:45:351
410000799522,17cyclictest1-21systemd19:25:001
410000799522,16cyclictest364-21snmpd22:36:521
410000299527,8cyclictest12-21ksoftirqd/021:33:240
4100002995224,13cyclictest4191323-21ssh00:09:100
410000299521,35cyclictest8222-21ssh00:31:460
4100011995142,5cyclictest4153796-21kworker/2:323:33:172
4100011995142,5cyclictest4074770-21kworker/2:221:14:072
4100011995141,6cyclictest4153796-21kworker/2:323:49:262
4100011995141,6cyclictest4074770-21kworker/2:221:16:452
4100011995140,6cyclictest4074770-21kworker/2:222:45:072
4100011995127,18cyclictest275-21dbus-daemon21:00:012
4100007995117,6cyclictest4102789-21sshd20:04:431
4100007995115,7cyclictest4103597-21sendmail-msp20:20:001
410000299515,15cyclictest5384-21ssh00:25:440
4100011995042,5cyclictest4074770-21kworker/2:222:43:392
4100011995041,6cyclictest4118078-21kworker/2:022:52:332
4100011995039,8cyclictest4074770-21kworker/2:220:55:012
4100011995039,7cyclictest4118078-21kworker/2:021:40:252
4100007995021,9cyclictest23-21ksoftirqd/120:41:451
4100007995015,5cyclictest0-21swapper/100:37:421
4100002995025,21cyclictest4137049-21latency_hist22:15:020
410001699492,5cyclictest364-21snmpd19:41:163
410001699492,5cyclictest364-21snmpd19:41:163
410001699491,43cyclictest364-21snmpd19:55:463
410001699491,43cyclictest364-21snmpd19:55:453
4100011994941,5cyclictest4191684-21kworker/2:200:34:012
4100011994941,5cyclictest4118078-21kworker/2:022:22:412
4100011994941,5cyclictest4074770-21kworker/2:222:11:112
4100011994939,7cyclictest4074770-21kworker/2:221:21:402
4100011994939,6cyclictest4074770-21kworker/2:222:00:012
4100011994938,7cyclictest4165625-21kworker/2:000:17:002
4100011994938,7cyclictest4153796-21kworker/2:323:15:012
4100007994915,6cyclictest4102655-21cat20:00:021
4100007994915,6cyclictest4102655-21cat20:00:021
4100007994914,9cyclictest23-21ksoftirqd/100:24:191
410000799491,19cyclictest4139437-21ssh22:20:011
4100002994914,9cyclictest12-21ksoftirqd/023:18:430
4100002994914,9cyclictest12-21ksoftirqd/023:18:430
4100011994840,5cyclictest4074770-21kworker/2:221:31:452
4100011994839,6cyclictest4153796-21kworker/2:323:43:062
4100011994839,6cyclictest4074770-21kworker/2:222:09:462
4100011994838,6cyclictest4191684-21kworker/2:200:35:552
4100011994838,6cyclictest4118078-21kworker/2:022:15:552
4100007994831,13cyclictest23-21ksoftirqd/121:00:001
4100002994820,24cyclictest12-21ksoftirqd/020:05:020
4100011994741,3cyclictest4153796-21kworker/2:322:57:422
4100011994741,3cyclictest4153796-21kworker/2:322:57:422
4100011994739,5cyclictest4165625-21kworker/2:000:13:512
4100011994738,5cyclictest4153796-21kworker/2:323:25:252
4100011994730,11cyclictest3467602-21systemd-journal20:40:002
4100011994730,11cyclictest3467602-21systemd-journal20:40:002
4100011994728,15cyclictest4132397-21latency_hist22:05:012
4100007994732,11cyclictest23-21ksoftirqd/119:40:021
4100002994722,10cyclictest4171400-21ssh23:26:520
410001699463,37cyclictest4100132-21latency_hist19:10:033
410001699463,37cyclictest4100132-21latency_hist19:10:023
4100011994639,4cyclictest4074770-21kworker/2:221:54:192
4100011994636,6cyclictest4074770-21kworker/2:221:27:212
410001199462,3cyclictest131rcu_preempt22:31:082
4100007994612,13cyclictest4100612-21cut19:20:021
4100007994612,13cyclictest4100612-21cut19:20:011
4100002994628,13cyclictest3467602-21systemd-journal20:00:010
4100002994628,13cyclictest3467602-21systemd-journal20:00:000
4100002994614,7cyclictest0-21swapper/020:10:000
4100011994537,5cyclictest4087111-21kworker/2:120:21:492
4100011994521,16cyclictest3467602-21systemd-journal20:20:002
4100007994531,9cyclictest309-21in:imuxsock19:41:141
4100007994531,9cyclictest309-21in:imuxsock19:41:131
4100002994531,11cyclictest12-21ksoftirqd/019:50:010
4100011994435,5cyclictest4104401-21munin-plugin-st20:35:002
4100011994424,13cyclictest275-21dbus-daemon19:30:012
4100011994413,20cyclictest4103051-21modprobe20:09:592
4100002994414,6cyclictest0-21swapper/020:50:480
4100011994335,5cyclictest4118078-21kworker/2:021:47:452
4100011994322,15cyclictest275-21dbus-daemon19:45:012
4100011994322,15cyclictest275-21dbus-daemon19:45:012
410000799435,4cyclictest131rcu_preempt21:10:021
4100007994312,5cyclictest309-21in:imuxsock20:50:001
4100002994332,8cyclictest12-21ksoftirqd/020:50:020
410000299431,38cyclictest4105113-21grep20:45:020
410000799426,15cyclictest275-21dbus-daemon20:55:011
4100007994232,7cyclictest309-21in:imuxsock19:55:001
410000799423,15cyclictest4102202-21cat19:50:021
4100002994224,13cyclictest275-21dbus-daemon19:15:010
410000299422,35cyclictest0-21swapper/020:40:000
410000299422,35cyclictest0-21swapper/020:39:590
4100011994132,5cyclictest4165625-21kworker/2:000:05:332
4100011994124,12cyclictest4106248-21systemd-run21:10:012
4100011994026,11cyclictest30-21ksoftirqd/223:20:342
4100002994030,6cyclictest12-21ksoftirqd/019:25:010
410000299402,34cyclictest4100690-21latency_hist19:20:030
4100002994011,6cyclictest0-21swapper/019:30:010
4100011993931,5cyclictest309-21in:imuxsock19:14:592
4100011993924,10cyclictest3467602-21systemd-journal19:56:132
4100011993924,10cyclictest3467602-21systemd-journal19:56:122
410000299393,7cyclictest4101774-21munin-run19:45:010
410000299393,7cyclictest4101774-21munin-run19:45:010
4100002993922,13cyclictest4100414-21sshd19:15:110
410001199382,3cyclictest131rcu_preempt20:30:022
410000299382,5cyclictest4105970-21munin-run21:05:010
410000299381,33cyclictest0-21swapper/020:30:000
4100002993720,13cyclictest4101222-21sshd19:33:130
410000299371,7cyclictest4100021-21latency_hist19:10:020
410000299371,7cyclictest4100021-21latency_hist19:10:010
410001199363,6cyclictest131rcu_preempt20:05:002
4100011993626,7cyclictest30-21ksoftirqd/219:25:012
4100011993624,10cyclictest30-21ksoftirqd/220:45:002
4100007993626,6cyclictest23-21ksoftirqd/119:15:021
410000299368,23cyclictest0-21swapper/019:35:170
4100011993517,13cyclictest275-21dbus-daemon19:40:012
4100007993524,8cyclictest23-21ksoftirqd/120:10:021
410000299354,8cyclictest0-21swapper/020:21:180
410001199343,25cyclictest4101243-21cut19:35:012
4100011993427,4cyclictest4074770-21kworker/2:221:05:022
4100011993425,5cyclictest309-21in:imuxsock19:55:002
410001199334,20cyclictest4103304-21modprobe20:15:002
4100007993323,7cyclictest23-21ksoftirqd/120:25:021
4100007992816,9cyclictest4100010-21sort19:10:021
4100007992816,9cyclictest4100010-21sort19:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional