You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 08:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jun 30, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2047899664,39cyclictest1416-21ssh21:25:152
2047899614,37cyclictest18258-21ssh22:10:142
20488995446,5cyclictest35966irq/58-eth022:00:143
2047899541,40cyclictest24777-21ssh23:25:132
2045799544,39cyclictest1368-21ssh23:30:110
2047899523,19cyclictest27346irq/40-dwc2_hso23:55:112
2046599526,10cyclictest131rcu_preempt20:10:131
2045799521,43cyclictest32483-21ssh22:00:150
2047899512,35cyclictest10467-21ssh23:35:122
2046599511,42cyclictest27078-21ssh00:20:151
2045799502,35cyclictest2554-21ssh22:55:120
2046599491,12cyclictest21-21ksoftirqd/122:25:141
2046599471,38cyclictest12793-21ssh00:30:141
2045799470,36cyclictest28588-21ssh23:45:130
2048899467,24cyclictest35966irq/58-eth023:00:133
2046599461,37cyclictest23219-21ssh00:00:121
2046599461,32cyclictest6597-21ssh23:15:151
2046599452,33cyclictest17911-21ssh00:15:091
2048899434,25cyclictest35966irq/58-eth022:50:163
2048899433,26cyclictest35966irq/58-eth021:55:133
2048899432,27cyclictest35966irq/58-eth023:40:123
2046599433,36cyclictest6329-21diskmemload23:10:121
2048899412,24cyclictest35966irq/58-eth019:15:143
20488994121,14cyclictest35966irq/58-eth021:10:143
20488994121,14cyclictest35966irq/58-eth021:10:133
2048899394,30cyclictest0-21swapper/322:55:123
2048899394,22cyclictest35966irq/58-eth000:10:153
2048899393,18cyclictest35966irq/58-eth023:30:103
2048899392,25cyclictest35966irq/58-eth000:25:143
20488993921,10cyclictest0-21swapper/323:35:123
20457993916,18cyclictest12-21ksoftirqd/021:15:150
2045799386,20cyclictest12-21ksoftirqd/021:05:130
2048899371,2cyclictest0-21swapper/319:50:113
2048899371,25cyclictest35966irq/58-eth023:20:123
20465993711,18cyclictest21-21ksoftirqd/123:50:121
2048899363,17cyclictest35966irq/58-eth022:20:123
2045799361,32cyclictest23189-21ssh00:00:100
2048899353,28cyclictest0-21swapper/321:15:143
2048899351,30cyclictest0-21swapper/300:30:123
2048899351,23cyclictest35966irq/58-eth021:35:143
2045799354,19cyclictest12-21ksoftirqd/021:00:140
2048899342,24cyclictest0-21swapper/300:00:113
2048899341,2cyclictest0-21swapper/323:55:113
2047899341,17cyclictest24825-21ssh21:20:132
2045799342,27cyclictest11670-21ssh23:00:120
2047899332,14cyclictest131rcu_preempt20:10:122
2046599333,17cyclictest25978-21ssh22:50:161
2045799333,22cyclictest15699-21ssh23:20:130
2048899324,24cyclictest0-21swapper/323:05:123
2048899323,22cyclictest0-21swapper/320:40:153
2045799322,27cyclictest27234-21ssh22:15:130
2048899313,24cyclictest0-21swapper/319:10:143
2048899313,19cyclictest0-21swapper/300:35:143
2048899311,4cyclictest0-21swapper/321:40:123
2047899314,16cyclictest27346irq/40-dwc2_hso21:35:122
2046599318,19cyclictest14340-21ssh21:50:161
2046599313,19cyclictest21-21ksoftirqd/120:55:151
2046599312,20cyclictest19499-21ssh23:40:111
2045799311,5cyclictest5186-21apt-get23:50:130
2048899303,23cyclictest0-21swapper/321:20:143
2048899302,2cyclictest0-21swapper/320:35:123
2048899302,2cyclictest0-21swapper/320:35:113
2048899301,3cyclictest0-21swapper/320:45:113
2048899301,17cyclictest0-21swapper/321:45:133
2048899301,17cyclictest0-21swapper/321:45:133
2048899293,18cyclictest0-21swapper/320:25:153
2048899292,2cyclictest0-21swapper/322:40:133
2048899291,2cyclictest0-21swapper/320:05:133
2047899294,19cyclictest27346irq/40-dwc2_hso00:20:142
2047899294,17cyclictest27346irq/40-dwc2_hso21:05:122
2047899292,14cyclictest27346irq/40-dwc2_hso22:35:122
2047899292,14cyclictest27346irq/40-dwc2_hso22:35:122
2046599296,16cyclictest21-21ksoftirqd/119:40:121
2045799295,20cyclictest151rcuc/021:40:130
2048899281,4cyclictest0-21swapper/322:05:143
2048899281,2cyclictest0-21swapper/321:25:133
2048899281,2cyclictest0-21swapper/320:15:103
2048899281,21cyclictest0-21swapper/322:25:133
2047899283,17cyclictest22103-21ssh22:30:132
2047899283,15cyclictest27346irq/40-dwc2_hso23:45:132
2047899283,15cyclictest27346irq/40-dwc2_hso19:55:122
2047899282,19cyclictest27346irq/40-dwc2_hso20:30:122
2047899282,19cyclictest0-21swapper/220:15:122
2047899282,16cyclictest27346irq/40-dwc2_hso19:15:132
2046599288,9cyclictest16766systemd-network21:25:021
2046599281,23cyclictest19435-21ssh21:35:121
2048899272,3cyclictest0-21swapper/321:05:123
2048899271,3cyclictest0-21swapper/321:50:143
2048899271,3cyclictest0-21swapper/300:15:123
2048899271,23cyclictest0-21swapper/323:15:143
2047899275,12cyclictest27346irq/40-dwc2_hso20:00:142
2047899273,20cyclictest27646-21ssh00:36:412
2047899273,13cyclictest27346irq/40-dwc2_hso20:40:122
2047899272,20cyclictest17264-21ssh00:14:422
2047899272,19cyclictest0-21swapper/219:20:152
20478992716,8cyclictest14325-21ssh21:50:142
2047899271,20cyclictest32485-21rm22:00:142
2046599273,17cyclictest21-21ksoftirqd/122:15:131
2045799272,19cyclictest930-21snmpd22:10:140
2048899262,6cyclictest0-21swapper/320:10:123
2048899261,2cyclictest0-21swapper/323:10:133
2047899264,14cyclictest27346irq/40-dwc2_hso20:45:132
2047899263,14cyclictest27346irq/40-dwc2_hso20:55:122
2047899263,14cyclictest27346irq/40-dwc2_hso00:00:122
2047899262,9cyclictest24689-21ssh00:17:412
2047899262,9cyclictest19652-21ssh00:32:582
2047899262,18cyclictest930-21snmpd00:25:412
2046599262,20cyclictest20792-21ssh23:05:141
2046599261,18cyclictest21-21ksoftirqd/100:35:141
2045799266,3cyclictest12-21ksoftirqd/022:45:120
2045799262,19cyclictest9153-21ssh22:05:130
20457992616,5cyclictest0-21swapper/023:55:110
2045799261,18cyclictest21783-21sh00:35:140
2048899251,3cyclictest0-21swapper/323:51:233
2048899251,3cyclictest0-21swapper/323:45:323
2048899251,3cyclictest0-21swapper/322:19:143
2048899251,3cyclictest0-21swapper/322:10:273
2048899251,3cyclictest0-21swapper/320:55:143
2048899251,2cyclictest0-21swapper/322:39:153
2048899251,2cyclictest0-21swapper/322:39:143
2048899251,2cyclictest0-21swapper/300:09:293
2048899251,21cyclictest0-21swapper/319:45:163
2048899251,21cyclictest0-21swapper/300:20:163
20488992511,9cyclictest35966irq/58-eth022:30:143
2048899250,5cyclictest0-21swapper/320:30:113
2047899252,19cyclictest0-21swapper/221:00:142
2047899252,13cyclictest25982-21ssh22:50:152
2047899250,21cyclictest9221-21ls22:05:142
2046599253,15cyclictest0-21swapper/119:50:121
20465992516,6cyclictest10076-21cron23:35:011
2048899241,5cyclictest0-21swapper/321:30:223
2048899241,3cyclictest0-21swapper/323:25:143
2048899241,3cyclictest0-21swapper/322:45:443
2048899241,3cyclictest0-21swapper/320:00:143
2047899249,8cyclictest614-21dbus-daemon22:17:382
2047899242,18cyclictest2717-21ssh21:42:342
2047899242,15cyclictest27346irq/40-dwc2_hso21:45:142
2047899242,15cyclictest27346irq/40-dwc2_hso21:45:132
2047899241,21cyclictest3123-21irqrtprio22:55:182
20465992412,10cyclictest0-21swapper/122:30:141
20465992411,8cyclictest21-21ksoftirqd/122:45:111
2046599241,15cyclictest131rcu_preempt21:55:211
2045799243,15cyclictest12-21ksoftirqd/020:45:110
2045799242,16cyclictest6756-21ssh21:10:150
2045799242,16cyclictest6756-21ssh21:10:140
20457992412,7cyclictest6329-21diskmemload23:35:130
2045799241,18cyclictest3021-21ssh00:24:410
2048899231,2cyclictest0-21swapper/319:40:163
2048899231,2cyclictest0-21swapper/319:35:223
2048899231,19cyclictest0-21swapper/319:20:163
2048899230,3cyclictest0-21swapper/319:55:143
2047899239,5cyclictest9808-21latency_hist20:40:012
2047899239,5cyclictest9808-21latency_hist20:40:012
2047899234,12cyclictest27346irq/40-dwc2_hso19:25:142
2047899232,11cyclictest930-21snmpd23:15:272
2047899231,8cyclictest1534166ssh22:25:232
20465992310,11cyclictest0-21swapper/121:41:551
2048899221,3cyclictest0-21swapper/320:50:333
2048899221,3cyclictest0-21swapper/320:20:163
2048899221,2cyclictest0-21swapper/321:00:133
2048899221,2cyclictest0-21swapper/319:30:213
2048899221,2cyclictest0-21swapper/319:25:253
2047899223,10cyclictest14486-21sh23:00:212
2047899221,15cyclictest1387-21ssh23:30:122
2047899221,11cyclictest13944-21latency_hist23:55:022
2046599222,15cyclictest15688-21apt-get23:20:121
20465992215,4cyclictest201rcuc/121:45:131
20465992215,4cyclictest201rcuc/121:45:121
20465992214,4cyclictest21-21ksoftirqd/100:25:121
2046599221,15cyclictest5133-21find20:35:121
2046599221,15cyclictest5133-21find20:35:111
2046599221,14cyclictest4843-21ls23:50:021
2045799225,10cyclictest22088-21ssh22:30:130
2045799223,14cyclictest7796-21ssh22:40:130
2045799221,3cyclictest131rcu_preempt23:10:130
20457992210,9cyclictest12-21ksoftirqd/022:50:140
2047899219,9cyclictest8533-21cron00:10:012
2047899216,13cyclictest4865-21kthreadcore22:20:192
2047899213,9cyclictest20669-21mailstats19:40:292
2047899212,16cyclictest13206-21kthreadcore19:35:192
20478992112,7cyclictest15816-21cpu21:15:162
2047899211,17cyclictest21167-21kthreadcore23:40:202
2047899211,17cyclictest10576-21grep21:30:152
2047899211,12cyclictest25223-21grep19:15:022
2047899211,10cyclictest27005-21grep19:50:172
2047899211,10cyclictest23838-21irqcore21:55:192
2046599212,13cyclictest9164-21ssh22:05:151
2046599210,17cyclictest14458-21ls23:55:161
2045799212,16cyclictest5235-21ssh21:45:140
2045799212,16cyclictest5235-21ssh21:45:130
2045799211,15cyclictest19523-21ssh23:40:140
2047899208,5cyclictest11639-21cut20:05:212
2047899206,5cyclictest26500-21apt-config19:50:012
2047899206,11cyclictest29018-21kthreadcore20:25:212
2047899203,8cyclictest15839-21ssh23:20:162
2047899203,14cyclictest11209-21timerwakeupswit19:30:292
2047899202,9cyclictest21505-21kthreadcore20:50:222
2047899202,10cyclictest22488-21ssh23:05:202
2047899201,9cyclictest7805-21ssh22:40:142
2047899201,9cyclictest23344-21fschecks_time20:20:182
2047899201,8cyclictest6699-21/usr/sbin/munin21:10:192
2047899201,8cyclictest6699-21/usr/sbin/munin21:10:192
2047899201,10cyclictest20490-21ntp_192.168.11422:45:252
2046599209,8cyclictest12334-21df_inode19:35:151
2046599202,12cyclictest8967-21ssh00:10:161
2046599201,3cyclictest131rcu_preempt23:37:091
2046599201,14cyclictest6742-21ssh21:10:141
2046599201,14cyclictest6742-21ssh21:10:131
2046599201,13cyclictest11129-21ls19:30:281
2046599200,16cyclictest25505-21switchtime19:45:271
2045799207,9cyclictest12-21ksoftirqd/021:30:140
2045799202,14cyclictest20781-21ssh23:05:130
2045799202,13cyclictest7483-21ntp_192.168.11422:20:240
20457992012,5cyclictest6329-21diskmemload00:10:260
2045799201,16cyclictest28797-21missed_timers19:15:240
2045799201,15cyclictest17936-21ssh00:15:110
2045799201,14cyclictest3716-21sh00:25:150
2045799200,15cyclictest31501-21ls19:55:010
2047899192,8cyclictest30125-21fschecks_count23:10:152
2046599199,7cyclictest2668-21df_inode22:55:141
2046599193,7cyclictest21-21ksoftirqd/123:26:111
2046599192,14cyclictest4051-21cpuspeed20:00:151
2046599192,12cyclictest32412-21ssh00:05:181
2046599191,3cyclictest131rcu_preempt21:30:241
20465991911,6cyclictest24839-21sshd22:12:381
2046599191,14cyclictest7559-21ssh22:20:241
2046599191,11cyclictest715-21irqrtprio22:00:181
2046599190,15cyclictest15205-21missed_timers23:00:231
2045799199,6cyclictest0-21swapper/020:15:120
2045799196,9cyclictest0-21swapper/019:40:130
2045799193,9cyclictest0-21swapper/023:25:130
2045799193,13cyclictest27356-21kthreadcore21:20:210
2045799192,14cyclictest19441-21ssh21:35:130
2045799191,3cyclictest0-21swapper/023:18:040
2045799191,11cyclictest0-21swapper/020:55:120
2048899181,5cyclictest0-21swapper/319:10:023
2047899181,14cyclictest20615-21latency_hist19:10:012
2046599186,7cyclictest20453-21cyclictest20:20:151
2046599183,12cyclictest2848-21sed22:35:311
2046599183,12cyclictest2848-21sed22:35:301
2046599181,2cyclictest131rcu_preempt20:45:161
2046599181,13cyclictest16029-21sort21:15:171
2046599181,11cyclictest2795-21ls19:55:271
2046599180,2cyclictest131rcu_preempt22:40:041
2045799189,4cyclictest630-21Async09:36:310
2045799188,7cyclictest11071-21kthreadcore20:05:220
2045799182,14cyclictest4652-21ssh21:25:230
2045799182,12cyclictest25637-21grep19:45:280
2045799182,11cyclictest1-21systemd00:32:580
2045799181,9cyclictest0-21swapper/022:29:580
20457991812,4cyclictest0-21swapper/019:20:120
20457991811,5cyclictest12018-21munin-plugin-st19:35:000
20457991811,5cyclictest0-21swapper/020:00:260
2045799181,11cyclictest6329-21diskmemload21:55:240
2046599175,9cyclictest2617-21wc19:25:011
2046599171,13cyclictest9548-21kthreadcore20:05:191
2046599171,13cyclictest26523-21kthreadcore19:15:211
2046599171,13cyclictest20587-21cat19:10:021
2046599171,12cyclictest6331-21latency_hist21:10:011
2046599171,12cyclictest21762-21netstat20:15:211
2046599170,9cyclictest29719-21irqcore21:00:181
2045799175,9cyclictest3382-21latency00:05:280
2045799174,10cyclictest27005-21unixbench_multi20:20:300
2045799172,11cyclictest0-21swapper/022:35:140
2045799172,11cyclictest0-21swapper/022:35:130
2045799171,14cyclictest14609-21ls21:50:180
2045799171,13cyclictest0-21swapper/019:35:150
2045799171,12cyclictest13683-21diskstats20:10:140
2045799171,11cyclictest2419-21latency19:55:220
2046599167,7cyclictest20844-21fw_packets19:10:171
2046599162,11cyclictest21016-21kthreadcore20:50:211
2046599162,10cyclictest0-21swapper/121:25:131
2046599161,12cyclictest6300-21switchtime19:25:291
2046599161,12cyclictest2926-21tr20:30:211
20465991610,4cyclictest0-21swapper/120:25:151
2046599160,13cyclictest14552-21latency_hist20:45:001
2045799167,7cyclictest501-21fw_packets20:30:150
2045799167,6cyclictest20107-21irqrtprio20:50:210
2045799164,6cyclictest0-21swapper/020:40:150
2045799162,11cyclictest6053-21ntp_192.168.11419:25:240
2045799162,10cyclictest0-21swapper/020:35:140
2045799162,10cyclictest0-21swapper/020:35:130
2045799162,10cyclictest0-21swapper/020:25:150
2045799141,9cyclictest20549-21grep19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional