You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-19 - 07:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Aug 19, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2563699250,24cyclictest0-21swapper/321:53:093
2563699250,24cyclictest0-21swapper/321:25:523
2562799250,24cyclictest5445-21ssh22:22:361
2562799250,24cyclictest25948-21ssh23:10:041
2562299250,24cyclictest19319-21diskmemload22:20:350
25636992423,0cyclictest0-21swapper/323:50:373
25636992422,1cyclictest0-21swapper/323:59:523
25636992422,1cyclictest0-21swapper/322:01:093
2563699240,23cyclictest0-21swapper/322:07:353
2563699240,23cyclictest0-21swapper/321:30:413
2563699240,23cyclictest0-21swapper/300:36:183
2563699240,23cyclictest0-21swapper/300:19:433
25631992423,1cyclictest19319-21diskmemload21:14:462
25631992422,1cyclictest0-21swapper/223:37:382
25631992422,1cyclictest0-21swapper/222:41:322
25631992420,0cyclictest0-21swapper/222:02:082
2563199240,24cyclictest0-21swapper/222:28:402
2563199240,23cyclictest21926-21rm00:31:232
2563199240,23cyclictest19319-21diskmemload23:20:482
2563199240,23cyclictest19319-21diskmemload00:11:172
25627992424,0cyclictest0-21swapper/122:58:171
2562799240,24cyclictest0-21swapper/123:17:471
2562799240,1cyclictest30168-21ssh21:18:581
2562799240,0cyclictest19319-21diskmemload21:10:101
2562799240,0cyclictest0-21swapper/122:13:291
25622992423,1cyclictest8445-21ssh23:22:430
25622992423,1cyclictest0-21swapper/023:07:460
25622992422,1cyclictest5841-21ssh23:49:430
25622992422,1cyclictest20115-21ssh21:10:070
2562299240,24cyclictest0-21swapper/021:29:070
2562299240,23cyclictest7063-21sh22:52:350
2562299240,23cyclictest19319-21diskmemload23:17:320
2562299240,23cyclictest19319-21diskmemload22:10:160
2562299240,23cyclictest12401-21ssh23:25:320
2562299240,23cyclictest0-21swapper/023:58:450
2562299240,1cyclictest14414-21ssh22:01:140
2562299240,0cyclictest0-21swapper/000:23:090
25636992323,0cyclictest0-21swapper/323:41:563
25636992323,0cyclictest0-21swapper/323:11:543
25636992323,0cyclictest0-21swapper/322:25:073
25636992322,0cyclictest0-21swapper/322:33:313
25636992321,1cyclictest0-21swapper/322:51:213
25636992321,1cyclictest0-21swapper/322:44:373
25636992320,2cyclictest1367-21diskmemload23:15:453
2563699231,21cyclictest27052-21diskmemload22:13:213
2563699230,23cyclictest0-21swapper/321:41:013
2563699230,23cyclictest0-21swapper/300:06:553
2563699230,22cyclictest32126-21diskmemload21:49:443
2563699230,22cyclictest215452sleep323:35:063
2563699230,22cyclictest20211-21latency00:30:103
2563699230,22cyclictest14752-21diskmemload00:25:103
2563699230,22cyclictest0-21swapper/322:56:123
2563699230,1cyclictest6243-21apt-get21:55:043
2563699230,1cyclictest0-21swapper/323:27:413
2563699230,0cyclictest19319-21diskmemload22:36:483
2563699230,0cyclictest0-21swapper/322:47:173
25631992323,0cyclictest19319-21diskmemload21:35:152
25631992323,0cyclictest0-21swapper/221:30:252
25631992323,0cyclictest0-21swapper/200:38:582
25631992322,1cyclictest31347-21apt-get21:20:042
25631992322,1cyclictest0-21swapper/223:06:512
25631992321,1cyclictest19294-21ssh23:03:562
25631992320,2cyclictest30752-21rm23:42:052
2563199231,21cyclictest0-21swapper/222:21:132
2563199230,2cyclictest10632-21ssh23:53:002
2563199230,23cyclictest0-21swapper/223:55:092
2563199230,23cyclictest0-21swapper/223:46:492
2563199230,23cyclictest0-21swapper/221:51:262
2563199230,23cyclictest0-21swapper/221:25:562
2563199230,22cyclictest2498-21perf20:35:012
2563199230,22cyclictest18732-21taskset22:35:052
2563199230,22cyclictest18426-21basename22:35:002
2563199230,22cyclictest15465-21ssh23:29:422
2563199230,22cyclictest0-21swapper/219:55:052
2563199230,1cyclictest0-21swapper/222:46:032
2563199230,1cyclictest0-21swapper/222:08:432
2563199230,0cyclictest28-21ksoftirqd/223:33:192
2563199230,0cyclictest19319-21diskmemload22:15:072
2563199230,0cyclictest0-21swapper/200:28:012
25627992323,0cyclictest19319-21diskmemload00:29:021
25627992323,0cyclictest0-21swapper/123:55:091
25627992323,0cyclictest0-21swapper/123:25:571
25627992323,0cyclictest0-21swapper/122:15:061
25627992323,0cyclictest0-21swapper/121:29:261
25627992322,1cyclictest13702-21ssh21:32:311
25627992321,1cyclictest10449-21ssh21:58:291
25627992320,2cyclictest25843-21ssh00:05:331
2562799230,2cyclictest23262-21rm21:40:211
2562799230,2cyclictest20304-21sh23:05:031
2562799230,2cyclictest19877-21ssh23:33:141
2562799230,23cyclictest19319-21diskmemload22:49:121
2562799230,23cyclictest0-21swapper/121:22:521
2562799230,22cyclictest7168-21df23:50:071
2562799230,22cyclictest5616-21cron19:35:011
2562799230,22cyclictest4794-21ssh23:20:011
2562799230,22cyclictest32286-21sh00:11:481
2562799230,22cyclictest22671-21ssh00:32:281
2562799230,22cyclictest15748-21rm22:31:201
2562799230,22cyclictest15633-21sed23:00:091
2562799230,22cyclictest0-21swapper/123:35:061
2562799230,0cyclictest19319-21diskmemload00:02:041
2562799230,0cyclictest0-21swapper/122:06:241
2562799230,0cyclictest0-21swapper/121:36:491
25622992323,0cyclictest19319-21diskmemload23:41:330
25622992323,0cyclictest19319-21diskmemload22:45:100
25622992323,0cyclictest0-21swapper/021:51:210
25622992323,0cyclictest0-21swapper/021:45:470
25622992323,0cyclictest0-21swapper/000:33:550
25622992321,1cyclictest0-21swapper/000:35:470
25622992320,3cyclictest0-21swapper/020:00:060
25622992320,2cyclictest22259-21sh22:09:210
25622992320,2cyclictest21838-21sh00:02:550
25622992320,2cyclictest19319-21diskmemload21:15:100
2562299231,1cyclictest62550irq/127-eno122:35:520
2562299230,23cyclictest19205-21ssh23:03:490
2562299230,23cyclictest0-21swapper/022:25:500
2562299230,23cyclictest0-21swapper/021:20:510
2562299230,22cyclictest25656-21ssh00:05:190
2562299230,22cyclictest19319-21diskmemload00:17:460
2562299230,21cyclictest6695-21ssh23:50:010
2562299230,1cyclictest19319-21diskmemload21:59:520
2562299230,1cyclictest0-21swapper/000:10:120
2562299230,0cyclictest0-21swapper/021:42:360
25636992222,0cyclictest0-21swapper/300:20:023
25636992221,1cyclictest19319-21diskmemload23:00:073
25636992221,1cyclictest0-21swapper/323:32:243
25636992221,1cyclictest0-21swapper/320:25:053
25636992220,1cyclictest0-21swapper/323:20:093
25636992220,1cyclictest0-21swapper/322:20:013
25636992220,1cyclictest0-21swapper/300:10:323
25636992219,2cyclictest29646-21sh22:15:113
2563699220,21cyclictest8305-21apt-get19:40:053
2563699220,21cyclictest0-21swapper/323:46:343
2563699220,21cyclictest0-21swapper/321:36:173
2563699220,21cyclictest0-21swapper/321:20:593
2563699220,21cyclictest0-21swapper/321:15:053
2563699220,21cyclictest0-21swapper/319:45:073
2563699220,21cyclictest0-21swapper/300:00:063
2563699220,1cyclictest3487-21apt-get19:30:023
2563699220,1cyclictest0-21swapper/321:12:003
25631992222,0cyclictest0-21swapper/200:17:082
25631992220,1cyclictest0-21swapper/200:24:042
2563199220,2cyclictest27650-21perf20:20:012
2563199220,21cyclictest21194-21rm00:01:582
2563199220,21cyclictest0-21swapper/222:56:382
2563199220,21cyclictest0-21swapper/222:11:222
2563199220,21cyclictest0-21swapper/221:55:022
2563199220,1cyclictest3466-21/usr/sbin/munin22:50:142
2563199220,1cyclictest28513-21ssh23:11:152
2563199220,1cyclictest28230-21ssh00:09:032
2563199220,1cyclictest0-21swapper/221:40:052
2563199220,0cyclictest0-21swapper/223:15:122
25627992222,0cyclictest0-21swapper/122:36:481
25627992222,0cyclictest0-21swapper/122:27:321
25627992221,1cyclictest24349-21apt-get22:40:061
25627992221,1cyclictest19319-21diskmemload21:53:091
25627992221,0cyclictest0-21swapper/119:45:071
25627992220,2cyclictest4451-21ssh23:47:481
25627992220,1cyclictest30050-21basename20:25:011
2562799220,21cyclictest5493-21ssh22:50:291
2562799220,21cyclictest32492-21cat20:30:011
2562799220,21cyclictest25125-21cstates00:35:061
2562799220,21cyclictest22819-21latency_hist20:10:001
2562799220,21cyclictest0-21swapper/119:15:001
2562799220,1cyclictest31784-21ssh21:49:111
2562799220,1cyclictest31544-21ssh23:43:121
2562799220,1cyclictest14016-21ssh22:00:421
2562799220,1cyclictest13041-21sh00:24:311
25622992222,0cyclictest0-21swapper/019:40:040
25622992220,1cyclictest30725-21ssh22:15:340
2562299220,2cyclictest15482-21/usr/sbin/munin00:25:150
2562299220,22cyclictest0-21swapper/023:31:560
2562299220,22cyclictest0-21swapper/022:58:030
2562299220,21cyclictest25941-21apt-get23:10:050
2562299220,21cyclictest0-21swapper/022:40:060
2562299220,21cyclictest0-21swapper/019:25:050
2562299220,1cyclictest19319-21diskmemload21:38:440
2562299220,1cyclictest15021-21sh22:30:250
25636992121,0cyclictest0-21swapper/319:50:013
25636992119,1cyclictest7309-21perf20:45:003
25636992118,2cyclictest18031-21awk20:00:003
2563699210,20cyclictest0-21swapper/323:06:073
2563699210,1cyclictest30051-21cat20:25:013
2563699210,0cyclictest2744-21apt-get20:35:053
2563699210,0cyclictest0-21swapper/320:00:063
2563199210,21cyclictest0-21swapper/219:15:052
2563199210,0cyclictest0-21swapper/219:10:032
25627992120,1cyclictest18304-21apt-get20:00:061
25627992120,1cyclictest0-21swapper/119:50:011
25627992119,1cyclictest14809-21apt-get21:00:061
2562799210,0cyclictest0-21swapper/100:15:181
25622992121,0cyclictest19319-21diskmemload23:36:270
25622992120,1cyclictest0-21swapper/020:15:010
2562299210,21cyclictest0-21swapper/019:30:020
25636992020,0cyclictest9753-21perf20:50:003
25636992020,0cyclictest0-21swapper/320:30:053
2563699200,0cyclictest0-21swapper/321:00:063
2563699200,0cyclictest0-21swapper/320:10:093
25631992020,0cyclictest0-21swapper/220:00:062
25631992019,1cyclictest0-21swapper/221:05:052
25631992019,0cyclictest0-21swapper/219:40:052
25631992018,1cyclictest5162-21apt-get20:40:062
25631992018,1cyclictest12816-21apt-get19:50:012
2563199200,0cyclictest23096-21apt-get20:10:102
25627992020,0cyclictest5890-21apt-get19:35:051
25627992020,0cyclictest0-21swapper/119:55:051
25627992020,0cyclictest0-21swapper/119:20:051
25627992019,1cyclictest25231-21perf20:15:011
2562799200,20cyclictest25486-21apt-get20:15:061
2562799200,0cyclictest0-21swapper/120:45:061
25622992020,0cyclictest0-21swapper/020:45:060
25622992020,0cyclictest0-21swapper/020:05:050
25622992020,0cyclictest0-21swapper/019:35:060
25622992020,0cyclictest0-21swapper/019:20:050
25622992018,1cyclictest4889-21perf20:40:000
2562299200,20cyclictest0-21swapper/019:15:050
2562299200,0cyclictest0-21swapper/020:30:050
25636991919,0cyclictest0-21swapper/319:20:053
25631991919,0cyclictest31055-21apt-get19:20:052
25631991919,0cyclictest0-21swapper/220:45:062
25631991919,0cyclictest0-21swapper/219:35:062
25631991918,0cyclictest30305-21apt-get20:25:042
25627991919,0cyclictest0-21swapper/120:35:051
2562299190,19cyclictest0-21swapper/019:45:070
25636991818,0cyclictest0-21swapper/320:05:053
25636991818,0cyclictest0-21swapper/319:30:013
25636991817,1cyclictest17205-21apt-get21:05:063
25636991817,0cyclictest0-21swapper/320:15:063
25627991818,0cyclictest0-21swapper/119:40:051
2562799180,18cyclictest0-21swapper/120:40:061
25622991818,0cyclictest0-21swapper/019:10:030
25631991716,1cyclictest27912-21apt-get20:20:062
25631991716,1cyclictest10702-21apt-get19:45:072
25627991717,0cyclictest12392-21apt-get20:55:051
25627991716,0cyclictest32756-21apt-get20:30:051
25622991717,0cyclictest0-21swapper/021:05:050
25622991717,0cyclictest0-21swapper/020:55:050
25622991717,0cyclictest0-21swapper/020:20:060
25622991717,0cyclictest0-21swapper/020:15:060
25622991717,0cyclictest0-21swapper/019:55:050
25636991615,1cyclictest0-21swapper/320:55:053
25631991616,0cyclictest0-21swapper/220:55:052
25631991614,1cyclictest21950irq/142-nvme0q320:55:012
25631991613,2cyclictest313-21systemd-journal21:05:002
2563199160,0cyclictest0-21swapper/220:05:052
2562799160,15cyclictest12117-21cron20:55:001
25622991614,1cyclictest783-21snmpd20:30:010
25636991512,3cyclictest0-21swapper/319:36:483
2563699150,1cyclictest0-21swapper/320:50:063
25631991513,1cyclictest0-21swapper/219:25:142
25622991513,1cyclictest606-21dbus-daemon20:45:000
2562299150,1cyclictest0-21swapper/020:50:060
25636991414,0cyclictest0-21swapper/319:15:053
25631991414,0cyclictest0-21swapper/220:35:052
25631991411,2cyclictest4665-21head19:30:082
2562799144,8cyclictest29366-21/usr/sbin/munin19:15:111
2562799140,14cyclictest0-21swapper/119:26:441
2562799140,13cyclictest0-21swapper/121:05:051
2562299140,14cyclictest0-21swapper/021:00:170
2562299140,14cyclictest0-21swapper/019:50:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional