You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 16:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Apr 20, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19339992720,6cyclictest8847-21apt-config12:15:013
1933999270,25cyclictest313-21systemd-journal11:35:013
1933999270,1cyclictest29051-21dump-pmu-power12:35:003
19339992622,3cyclictest14295-21diskmemload09:37:513
19325992624,2cyclictest0-21swapper/012:29:520
19325992623,2cyclictest0-21swapper/011:27:450
19325992620,5cyclictest11090-21rm11:13:360
1932599260,25cyclictest0-21swapper/011:49:450
1932599260,21cyclictest14295-21diskmemload12:15:010
1932599260,1cyclictest0-21swapper/011:35:010
1933999250,1cyclictest0-21swapper/310:27:113
19328992523,1cyclictest29389-21ssh12:02:591
1932899250,2cyclictest26262-21rm11:27:441
1932899250,24cyclictest19253-21ssh09:14:421
19325992523,2cyclictest0-21swapper/010:23:020
19325992521,3cyclictest5937-21sh10:35:180
19325992521,3cyclictest0-21swapper/011:01:010
1932599250,24cyclictest0-21swapper/012:08:520
1932599250,21cyclictest28415-21sh10:25:380
1932599250,21cyclictest18658-21ssh11:20:120
1932599250,1cyclictest8436-21ssh10:06:030
1932599250,1cyclictest0-21swapper/012:35:530
1932599250,0cyclictest2123-21ssh09:29:070
1932599250,0cyclictest0-21swapper/011:38:220
1932599250,0cyclictest0-21swapper/011:16:290
1932599250,0cyclictest0-21swapper/009:24:150
19339992420,3cyclictest0-21swapper/312:25:383
19339992420,3cyclictest0-21swapper/312:06:033
19339992420,3cyclictest0-21swapper/311:44:273
19339992420,3cyclictest0-21swapper/311:05:543
19339992419,4cyclictest0-21swapper/311:55:073
1933999240,24cyclictest14295-21diskmemload11:35:033
1933999240,23cyclictest31294-21sh09:57:053
1933999240,23cyclictest11703-21apt-get08:00:063
1933999240,23cyclictest10897-21perf10:10:003
1933999240,23cyclictest0-21swapper/311:50:243
1933999240,23cyclictest0-21swapper/311:01:043
1933999240,23cyclictest0-21swapper/310:56:463
1933999240,23cyclictest0-21swapper/310:21:303
1933999240,20cyclictest0-21swapper/311:16:293
1933999240,20cyclictest0-21swapper/309:15:143
19333992423,1cyclictest9129-21ssh10:40:012
1933399240,24cyclictest14295-21diskmemload12:00:332
1933399240,23cyclictest15150-21ssh09:40:342
1933399240,1cyclictest31223-21sh11:32:422
1933399240,1cyclictest26998-21cron08:30:012
1933399240,0cyclictest0-21swapper/211:58:452
19328992424,0cyclictest0-21swapper/111:52:261
19328992423,1cyclictest14295-21diskmemload11:39:041
19328992421,2cyclictest974-21ssh09:27:161
19328992421,2cyclictest21793-21cron08:20:011
1932899240,23cyclictest6166-21sh12:11:231
19325992422,2cyclictest0-21swapper/009:54:420
1932599242,21cyclictest62550irq/127-eno112:12:140
19325992422,1cyclictest3802-21cron10:35:010
19325992422,1cyclictest14295-21diskmemload11:44:010
19325992421,2cyclictest13578-21sh10:44:250
19325992420,3cyclictest0-21swapper/012:03:100
19325992420,3cyclictest0-21swapper/009:47:180
19325992419,4cyclictest17546-21ssh10:47:400
1932599241,23cyclictest62550irq/127-eno110:59:020
1932599240,24cyclictest0-21swapper/010:54:300
1932599240,23cyclictest4730-21sh10:03:110
1932599240,23cyclictest22454-21ssh09:16:410
1932599240,23cyclictest14295-21diskmemload12:21:010
1932599240,23cyclictest0-21swapper/010:15:350
1932599240,23cyclictest0-21swapper/009:38:250
1932599240,1cyclictest0-21swapper/009:55:130
1932599240,0cyclictest14295-21diskmemload11:05:540
19339992321,1cyclictest14864-21cstates09:10:073
19339992320,3cyclictest0-21swapper/310:30:253
19339992320,3cyclictest0-21swapper/309:20:033
19339992320,2cyclictest14295-21diskmemload09:41:003
19339992320,2cyclictest10317-21rm10:40:113
19339992318,5cyclictest0-21swapper/311:20:053
1933999230,2cyclictest28049-21dump-pmu-power09:55:013
1933999230,2cyclictest16599-21ssh12:21:183
1933999230,23cyclictest0-21swapper/312:00:083
1933999230,23cyclictest0-21swapper/310:10:273
1933999230,22cyclictest16325-21sh10:45:403
1933999230,22cyclictest0-21swapper/312:11:463
1933999230,22cyclictest0-21swapper/310:15:523
1933999230,22cyclictest0-21swapper/309:25:513
1933999230,0cyclictest0-21swapper/310:38:043
19333992323,0cyclictest0-21swapper/212:38:102
19333992323,0cyclictest0-21swapper/212:17:362
19333992323,0cyclictest0-21swapper/209:11:592
19333992322,1cyclictest19222-1kworker/u9:110:56:342
19333992321,1cyclictest23480-21ssh09:18:242
19333992320,2cyclictest18295-21ssh10:15:382
1933399230,23cyclictest8068-21ssh12:14:332
1933399230,23cyclictest14295-21diskmemload10:30:252
1933399230,23cyclictest0-21swapper/212:33:122
1933399230,23cyclictest0-21swapper/211:47:152
1933399230,23cyclictest0-21swapper/211:08:312
1933399230,23cyclictest0-21swapper/211:00:452
1933399230,23cyclictest0-21swapper/210:45:212
1933399230,22cyclictest8015-21ssh09:35:022
1933399230,22cyclictest2122-21ssh12:07:592
1933399230,22cyclictest14295-21diskmemload09:47:372
1933399230,22cyclictest0-21swapper/210:11:192
1933399230,0cyclictest14295-21diskmemload11:38:222
1933399230,0cyclictest14295-21diskmemload09:54:352
1933399230,0cyclictest0-21swapper/211:50:302
1933399230,0cyclictest0-21swapper/209:31:312
1933399230,0cyclictest0-21swapper/209:27:362
19328992323,0cyclictest14295-21diskmemload11:47:201
19328992323,0cyclictest0-21swapper/112:36:221
19328992323,0cyclictest0-21swapper/112:31:281
19328992323,0cyclictest0-21swapper/111:18:111
19328992323,0cyclictest0-21swapper/111:04:101
19328992323,0cyclictest0-21swapper/109:49:481
19328992323,0cyclictest0-21swapper/109:34:361
19328992323,0cyclictest0-21swapper/109:15:571
19328992323,0cyclictest0-21swapper/107:10:051
1932899232,21cyclictest30132-21kworker/1:309:53:501
19328992321,1cyclictest0-21swapper/111:59:271
19328992320,2cyclictest16985-21ssh12:21:531
1932899230,23cyclictest0-21swapper/112:16:211
1932899230,23cyclictest0-21swapper/111:41:551
1932899230,23cyclictest0-21swapper/110:44:091
1932899230,22cyclictest3295-21sh10:00:531
1932899230,22cyclictest28880-21fschecks_time09:55:071
1932899230,22cyclictest27496-21sh09:20:541
1932899230,22cyclictest24390-21rm10:55:031
1932899230,22cyclictest23803-21ssh10:21:171
1932899230,22cyclictest21105-21vmstat10:50:151
1932899230,22cyclictest17933-21fschecks_count11:20:071
1932899230,0cyclictest0-21swapper/112:06:461
1932899230,0cyclictest0-21swapper/111:11:021
19325992323,0cyclictest0-21swapper/011:50:060
19325992321,2cyclictest0-21swapper/011:59:260
19325992321,1cyclictest24888-21apt-get08:25:050
19325992321,1cyclictest17279-21apt-get08:10:060
19325992321,1cyclictest17279-21apt-get08:10:060
19325992320,3cyclictest0-21swapper/010:12:080
19325992320,2cyclictest3068-21sh09:30:070
1932599230,22cyclictest0-21swapper/009:10:070
1932599230,0cyclictest14295-21diskmemload12:34:590
19339992220,1cyclictest5909-21ssh09:31:583
19339992216,5cyclictest29637-21cstates12:35:083
1933999220,21cyclictest27019-21apt-get08:30:023
1933999220,21cyclictest14295-21diskmemload09:45:253
1933999220,21cyclictest10790-21apt-get11:45:063
19333992222,0cyclictest0-21swapper/211:44:182
19333992222,0cyclictest0-21swapper/210:50:312
19333992222,0cyclictest0-21swapper/210:26:062
19333992220,1cyclictest0-21swapper/210:01:462
19333992220,1cyclictest0-21swapper/209:55:142
1933399220,22cyclictest0-21swapper/211:26:232
1933399220,22cyclictest0-21swapper/211:24:322
1933399220,21cyclictest28894-21munin-run07:30:012
1933399220,21cyclictest25232-21apt-get09:20:062
1933399220,21cyclictest23316-21ssh10:20:332
1933399220,21cyclictest14743-21fw_packets12:20:082
1933399220,21cyclictest0-21swapper/210:35:062
1933399220,1cyclictest31292-21munin-run07:35:012
1933399220,1cyclictest14550-21sh11:15:582
1933399220,0cyclictest14295-21diskmemload11:11:032
19328992222,0cyclictest14295-21diskmemload11:06:061
19328992222,0cyclictest0-21swapper/110:39:571
19328992222,0cyclictest0-21swapper/110:31:041
19328992220,1cyclictest28479-21sh10:25:481
1932899220,22cyclictest0-21swapper/111:30:581
1932899220,22cyclictest0-21swapper/110:06:201
1932899220,22cyclictest0-21swapper/109:35:201
1932899220,21cyclictest13825-21apt-get08:05:021
1932899220,21cyclictest0-21swapper/112:25:521
1932899220,1cyclictest16161-21ssh10:15:041
19325992221,0cyclictest29143-21apt-get07:30:060
1932599220,22cyclictest0-21swapper/009:40:300
1932599220,1cyclictest30079-21apt-get08:35:060
19339992121,0cyclictest0-21swapper/310:00:203
19339992118,2cyclictest21848-21perf08:20:013
1933999210,21cyclictest0-21swapper/308:45:063
1933999210,20cyclictest0-21swapper/308:35:063
1933999210,20cyclictest0-21swapper/308:20:053
1933999210,20cyclictest0-21swapper/308:10:063
1933999210,20cyclictest0-21swapper/308:10:063
1933999210,20cyclictest0-21swapper/308:05:053
1933999210,1cyclictest0-21swapper/307:25:063
19333992120,0cyclictest9790-21apt-get09:00:062
19333992120,0cyclictest0-21swapper/208:55:032
19333992120,0cyclictest0-21swapper/208:35:062
19333992120,0cyclictest0-21swapper/208:00:062
19333992119,1cyclictest7125-21munin-run08:55:012
19333992119,1cyclictest0-21swapper/208:00:002
1933399210,21cyclictest12186-21apt-get09:05:062
1933399210,21cyclictest0-21swapper/212:25:132
1933399210,20cyclictest0-21swapper/208:40:062
1933399210,1cyclictest0-21swapper/208:15:052
1933399210,0cyclictest14491-21apt-get08:05:052
19328992121,0cyclictest0-21swapper/110:45:041
19328992121,0cyclictest0-21swapper/110:10:241
19328992121,0cyclictest0-21swapper/107:55:051
1932899210,20cyclictest2529-21apt-get08:45:061
1932899210,20cyclictest0-21swapper/107:30:051
1932899210,0cyclictest0-21swapper/109:40:051
1932899210,0cyclictest0-21swapper/109:00:061
1932899210,0cyclictest0-21swapper/108:30:031
19325992120,1cyclictest19679-21apt-get08:15:050
19325992120,1cyclictest0-21swapper/008:20:050
19325992118,2cyclictest13845-21grep08:05:000
1932599210,0cyclictest21935-21apt-get07:15:050
19339992018,1cyclictest6889-21apt-get07:50:063
19333992020,0cyclictest24354-21apt-get07:20:052
19333992020,0cyclictest0-21swapper/207:45:052
19333992020,0cyclictest0-21swapper/207:15:052
19333992018,1cyclictest21819-21apt-get08:20:022
1933399200,0cyclictest0-21swapper/207:50:062
19328992020,0cyclictest1696-21apt-get07:40:051
19328992020,0cyclictest0-21swapper/107:45:011
1932899200,20cyclictest0-21swapper/108:35:061
1932899200,20cyclictest0-21swapper/108:00:061
1932899200,0cyclictest0-21swapper/107:15:051
19325992020,0cyclictest0-21swapper/008:30:020
19325992020,0cyclictest0-21swapper/008:05:020
19325992020,0cyclictest0-21swapper/007:50:060
19325992020,0cyclictest0-21swapper/007:10:040
19339991919,0cyclictest0-21swapper/309:05:063
19339991919,0cyclictest0-21swapper/308:25:053
19339991919,0cyclictest0-21swapper/307:30:063
19339991917,1cyclictest32550-21apt-get08:40:063
19339991917,1cyclictest0-21swapper/307:40:053
19339991914,0cyclictest0-21swapper/307:15:053
19333991919,0cyclictest19541-21apt-get07:10:042
19333991919,0cyclictest0-21swapper/208:10:062
19333991919,0cyclictest0-21swapper/208:10:062
19328991919,0cyclictest0-21swapper/108:25:011
19328991918,1cyclictest17239-21apt-get08:10:021
19328991918,1cyclictest17239-21apt-get08:10:021
19328991916,2cyclictest609-21rs:main1
19325991919,0cyclictest0-21swapper/009:00:060
19325991919,0cyclictest0-21swapper/008:40:060
19325991919,0cyclictest0-21swapper/007:55:060
19325991919,0cyclictest0-21swapper/007:40:050
1932599190,18cyclictest0-21swapper/007:25:060
19339991818,0cyclictest0-21swapper/307:10:043
19333991818,0cyclictest0-21swapper/208:45:062
19333991818,0cyclictest0-21swapper/207:40:052
19328991818,0cyclictest0-21swapper/109:05:071
19328991818,0cyclictest0-21swapper/108:55:031
19328991818,0cyclictest0-21swapper/107:35:061
19328991818,0cyclictest0-21swapper/107:20:051
19339991717,0cyclictest0-21swapper/309:00:063
1933999170,16cyclictest0-21swapper/307:55:053
1933999170,12cyclictest313-21systemd-journal07:40:013
19325991715,1cyclictest4709-21perf08:50:000
19333991616,0cyclictest0-21swapper/208:30:022
19328991616,0cyclictest0-21swapper/107:25:061
19339991514,1cyclictest0-21swapper/307:45:263
1933999150,14cyclictest8416-21ntp_states08:55:093
19333991513,1cyclictest32146-21perl07:35:092
19328991513,1cyclictest211rcuc/108:30:011
19325991514,1cyclictest24354-21apt-get07:20:050
19325991514,1cyclictest0-21swapper/007:35:070
1933999140,14cyclictest0-21swapper/308:50:343
1933999140,14cyclictest0-21swapper/307:21:143
19328991414,0cyclictest0-21swapper/107:50:061
1932899140,0cyclictest0-21swapper/108:50:591
19325991414,0cyclictest0-21swapper/009:05:300
1932599140,14cyclictest0-21swapper/008:50:150
1932599140,14cyclictest0-21swapper/007:45:160
1932599140,0cyclictest0-21swapper/008:55:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional