You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 07:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 29, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3102599846,5cyclictest131rcu_preempt20:40:130
31052997742,22cyclictest35966irq/58-eth000:25:133
31033997420,11cyclictest131rcu_preempt20:45:121
3103399664,47cyclictest26673-21ssh00:15:121
3102599646,41cyclictest2807-21ssh22:50:150
3103399637,14cyclictest21-21ksoftirqd/119:30:131
31052996254,4cyclictest35966irq/58-eth021:45:133
3104499624,28cyclictest8223-21ssh22:35:112
3102599613,45cyclictest6461-21ssh21:40:110
3104499604,43cyclictest3174-21ssh00:20:122
3104499604,43cyclictest3174-21ssh00:20:112
3102599554,47cyclictest17610-21ssh00:10:150
3103399545,38cyclictest24622-21ssh23:20:131
3103399545,38cyclictest24622-21ssh23:20:131
3105299494,32cyclictest35966irq/58-eth021:30:123
31052994927,13cyclictest35966irq/58-eth000:15:123
3102599492,40cyclictest26257-21ssh22:45:140
3102599473,36cyclictest26338-21ssh21:15:120
3102599472,34cyclictest28293-21ssh23:40:120
3104499454,24cyclictest15431-21ssh21:45:122
3102599452,37cyclictest14012-21ssh23:50:130
3103399435,25cyclictest13644-21ssh22:20:131
3105299413,26cyclictest35966irq/58-eth000:35:123
31044994115,19cyclictest27-21ksoftirqd/221:40:132
3102599412,17cyclictest8636-21ssh00:05:130
3102599403,27cyclictest3184-21ssh00:20:110
3102599403,27cyclictest3184-21ssh00:20:110
3104499392,30cyclictest27178-21cat23:55:302
3102599392,6cyclictest0-21swapper/022:43:290
3104499381,21cyclictest17181-21ssh22:40:122
3103399382,32cyclictest2889-21ssh21:20:121
31033993822,10cyclictest0-21swapper/100:00:131
3102599382,24cyclictest24411-21ssh21:50:110
3104499374,25cyclictest0-21swapper/222:50:152
3103399374,25cyclictest4955-21sh23:45:111
3102599374,20cyclictest12114-21ssh00:25:140
3105299365,22cyclictest35966irq/58-eth023:40:123
3105299362,22cyclictest35966irq/58-eth022:15:123
3104499362,28cyclictest15967-21awk22:55:292
3104499362,18cyclictest20455-21ssh23:52:262
31033993515,15cyclictest21-21ksoftirqd/100:10:141
3102599358,5cyclictest31472-21date22:30:020
3102599352,10cyclictest3892-21kthreadcore23:25:220
31044993416,15cyclictest27-21ksoftirqd/223:25:132
3102599342,17cyclictest15440-21ssh21:45:130
31025993410,4cyclictest4966-21ssh22:15:170
3105299333,20cyclictest35966irq/58-eth000:00:153
3103399333,23cyclictest9998-21ssh22:00:131
3102599331,8cyclictest31158-21apt-get19:10:120
3102599331,29cyclictest22996-21sh23:55:140
3102599331,11cyclictest13701-21missed_timers23:30:220
3102599331,10cyclictest27669-21wc22:10:010
3105299326,17cyclictest35966irq/58-eth022:50:143
3105299324,15cyclictest35966irq/58-eth019:20:123
3104499325,18cyclictest27346irq/40-dwc2_hso23:15:122
31044993221,8cyclictest0-21swapper/221:30:132
31044993220,6cyclictest28299-21ssh23:40:122
3103399324,17cyclictest29933-21ssh23:05:141
3102599321,11cyclictest3081-21ssh21:55:200
3102599321,11cyclictest17727-21munin-run19:30:010
3102599321,11cyclictest1068-21kthreadcore22:30:210
3102599320,11cyclictest15884-21ssh21:25:290
3105299313,21cyclictest35966irq/58-eth021:20:103
31052993110,9cyclictest35966irq/58-eth021:25:133
3104499316,17cyclictest27346irq/40-dwc2_hso20:20:122
3104499305,17cyclictest27346irq/40-dwc2_hso20:05:102
3102599301,12cyclictest30462-21irqcore00:35:170
3102599301,11cyclictest10911-21cut22:00:190
3104499293,16cyclictest27346irq/40-dwc2_hso19:10:112
3103399295,13cyclictest21-21ksoftirqd/119:55:131
3102599292,8cyclictest16910-21ls22:40:010
3102599292,15cyclictest4956-21rm23:45:110
3102599291,9cyclictest28209-21awk23:20:240
3102599291,9cyclictest28209-21awk23:20:240
3102599291,10cyclictest30001-21df21:35:140
3105299283,18cyclictest0-21swapper/323:52:263
3105299283,16cyclictest0-21swapper/323:55:293
31052992813,10cyclictest35966irq/58-eth021:10:133
3105299281,13cyclictest35966irq/58-eth023:00:143
3104499283,16cyclictest27346irq/40-dwc2_hso19:45:122
3104499282,22cyclictest930-21snmpd22:00:132
3104499281,21cyclictest26536-21sh00:15:052
3103399284,12cyclictest31014-21cyclictest19:45:121
3103399281,18cyclictest1178-21rm23:25:121
3102599282,6cyclictest0-21swapper/022:55:290
3105299271,12cyclictest35966irq/58-eth019:10:113
3104499275,16cyclictest27346irq/40-dwc2_hso23:00:132
3104499273,19cyclictest0-21swapper/200:26:402
3104499272,16cyclictest27346irq/40-dwc2_hso21:35:132
3103399279,12cyclictest630-21Async09:36:311
3103399272,13cyclictest11955-21ssh21:25:131
3102599273,6cyclictest0-21swapper/022:23:290
3105299269,11cyclictest35966irq/58-eth000:05:133
31052992612,9cyclictest35966irq/58-eth000:10:143
3104499263,16cyclictest27346irq/40-dwc2_hso22:20:122
3104499262,15cyclictest27346irq/40-dwc2_hso23:10:142
3104499261,16cyclictest27346irq/40-dwc2_hso19:30:132
3103399264,10cyclictest22616-21ssh22:25:141
3103399263,16cyclictest21-21ksoftirqd/121:05:121
3103399262,19cyclictest18371-21sh22:57:291
3102599261,19cyclictest21058-21ssh00:30:130
3105299252,15cyclictest0-21swapper/322:38:293
31052992515,7cyclictest35966irq/58-eth021:40:113
31052992512,4cyclictest35966irq/58-eth022:20:133
3105299251,14cyclictest35966irq/58-eth020:25:133
3105299251,14cyclictest35966irq/58-eth020:25:133
3104499252,16cyclictest0-21swapper/222:45:052
3105299244,14cyclictest0-21swapper/321:35:143
31052992419,3cyclictest321rcuc/319:25:133
31052992410,7cyclictest35966irq/58-eth022:40:123
3104499248,13cyclictest2419-21cron20:55:022
3104499246,12cyclictest27346irq/40-dwc2_hso20:55:132
3104499245,14cyclictest27346irq/40-dwc2_hso21:10:132
3104499243,6cyclictest0-21swapper/221:23:302
3104499243,17cyclictest27346irq/40-dwc2_hso22:05:132
3104499241,15cyclictest19600-21cut23:35:182
3104499240,16cyclictest13300-21cut19:25:162
3103399242,18cyclictest16901-21diskmemload23:55:141
31025992416,4cyclictest0-21swapper/021:10:130
3102599241,14cyclictest8157-21apt-get00:05:020
3105299237,9cyclictest35966irq/58-eth022:05:143
3105299232,12cyclictest0-21swapper/323:25:143
3104499232,6cyclictest0-21swapper/200:35:402
3104499232,6cyclictest0-21swapper/200:33:392
3104499232,15cyclictest0-21swapper/222:10:122
3104499231,8cyclictest4581-21ssh22:15:122
31044992312,9cyclictest30840-21ssh21:52:272
31044992312,9cyclictest19163-21ssh21:28:212
31044992311,9cyclictest30460-21sshd21:15:292
3103399231,19cyclictest0-21swapper/100:05:131
3102599231,15cyclictest20954-21ssh21:30:130
3105299222,15cyclictest35966irq/58-eth021:55:123
31052992211,8cyclictest0-21swapper/323:11:333
3104499228,6cyclictest1154-21fschecks_count21:55:152
3104499225,9cyclictest13969-21kthreadcore20:30:222
3104499224,9cyclictest7686-21fschecks_count21:00:162
3104499222,14cyclictest27346irq/40-dwc2_hso23:30:112
3104499221,18cyclictest3657-21grep19:15:152
31044992211,5cyclictest31596-21ssh22:30:112
3104499220,5cyclictest24241-21ls20:40:432
3103399228,4cyclictest21-21ksoftirqd/123:00:121
3103399222,3cyclictest26306-21ssh00:31:011
3103399222,14cyclictest3396-21ssh00:36:391
31033992214,5cyclictest211ksoftirqd/120:15:131
31033992212,8cyclictest21-21ksoftirqd/122:10:111
31033992212,4cyclictest21-21ksoftirqd/120:05:121
31033992210,10cyclictest21-21ksoftirqd/123:40:011
3105299212,12cyclictest0-21swapper/320:15:153
31052992116,3cyclictest321rcuc/323:17:303
31052992116,3cyclictest321rcuc/322:33:143
31052992116,3cyclictest321rcuc/319:15:143
3105299211,5cyclictest0-21swapper/320:00:123
31052992115,4cyclictest321rcuc/323:07:173
31052992112,6cyclictest35966irq/58-eth022:25:143
3104499212,16cyclictest8174-21kthreadcore20:25:212
3104499212,16cyclictest8174-21kthreadcore20:25:212
3104499211,9cyclictest32211-21grep00:00:162
31044992111,8cyclictest27-21ksoftirqd/219:50:302
31044992110,8cyclictest27778-21fschecks_count19:40:152
3103399211,17cyclictest6721-21grep23:10:151
3102599213,10cyclictest8626-21ssh23:10:200
3102599211,16cyclictest20934-21sh23:00:140
3102599210,15cyclictest8564-21idleruntime19:20:170
3105299204,10cyclictest0-21swapper/300:30:133
31052992015,3cyclictest321rcuc/323:46:573
31052992015,3cyclictest321rcuc/323:36:023
31052992015,3cyclictest321rcuc/320:35:253
3104499208,9cyclictest32590-21timerandwakeup20:15:292
3104499202,5cyclictest0-21swapper/222:25:142
3104499202,10cyclictest26594-21kthreadcore20:45:172
3104499201,9cyclictest19854-21tr00:10:222
3104499201,8cyclictest6603-21kthreadcore23:45:192
31044992013,5cyclictest14184-21sshd19:55:472
3104499201,11cyclictest930-21snmpd00:06:072
3104499201,10cyclictest17808-21kthreadcore20:35:202
3104499201,10cyclictest1298-21ps23:05:282
3104499201,10cyclictest11661-21missed_timers19:20:232
3104499200,14cyclictest24697-21kthreadcore19:35:212
3103399209,8cyclictest22621-21/usr/sbin/munin22:05:251
3103399208,3cyclictest0-21swapper/120:00:181
3103399205,8cyclictest31014-21cyclictest19:50:121
3103399204,3cyclictest131rcu_preempt21:40:171
3103399202,14cyclictest6415-21ntp_192.168.11422:50:261
3103399202,12cyclictest31289-21grep19:10:141
31033992015,3cyclictest0-21swapper/122:15:241
31033992013,4cyclictest28724-21usage_Anybus21:50:291
31033992013,4cyclictest21-21ksoftirqd/123:15:151
31033992012,5cyclictest24654-21open_inodes21:30:271
31033992011,6cyclictest30390-21grep22:45:321
3103399201,14cyclictest20090-21kthreadcore22:40:211
3103399201,14cyclictest13145-21apt-get19:25:121
31033992010,7cyclictest21-21ksoftirqd/123:30:111
3102599205,9cyclictest0-21swapper/019:55:140
31025992010,7cyclictest12112-21cron21:05:010
3105299193,9cyclictest0-21swapper/322:57:293
3105299193,11cyclictest0-21swapper/322:01:293
3105299192,11cyclictest0-21swapper/321:00:133
31052991914,3cyclictest321rcuc/323:24:363
31052991914,3cyclictest321rcuc/323:24:353
31052991914,3cyclictest321rcuc/322:49:203
3105299191,13cyclictest0-21swapper/321:05:143
3105299191,11cyclictest0-21swapper/320:30:153
31052991910,6cyclictest35966irq/58-eth023:30:123
3104499193,13cyclictest27346irq/40-dwc2_hso21:05:122
3104499191,9cyclictest26138-21kthreadcore20:10:202
3104499191,8cyclictest14774-21ps20:00:182
3104499191,15cyclictest25774-21kthreadcore23:20:212
3104499191,15cyclictest25774-21kthreadcore23:20:202
3103399197,9cyclictest8080-21cron19:20:021
3103399196,8cyclictest536-21usb-storage20:55:051
3103399193,3cyclictest131rcu_preempt20:50:131
3103399192,14cyclictest15688-21ntp_192.168.11400:25:251
31033991913,4cyclictest0-21swapper/121:00:161
31033991912,4cyclictest14256-21fschecks_time23:50:171
3103399191,15cyclictest3271-21timerwakeupswit22:30:291
3103399191,15cyclictest26876-21cut20:10:211
31033991911,5cyclictest0-21swapper/119:35:261
3103399191,14cyclictest29906-21ssh21:35:131
3103399191,13cyclictest20746-21ssh21:10:221
3103399191,10cyclictest21895-21tr20:40:351
3103399190,14cyclictest8618-21cut22:35:181
3102599191,14cyclictest29922-21ssh23:05:140
3102599191,14cyclictest19321-21ssh23:35:140
3105299182,10cyclictest0-21swapper/319:45:143
31052991813,3cyclictest0-21swapper/322:10:173
31052991813,3cyclictest0-21swapper/321:19:343
31052991812,4cyclictest0-21swapper/319:35:243
31052991812,4cyclictest0-21swapper/300:24:393
31052991812,4cyclictest0-21swapper/300:24:393
3105299181,10cyclictest0-21swapper/320:50:163
31052991810,6cyclictest0-21swapper/321:54:053
31052991810,6cyclictest0-21swapper/320:55:183
3103399188,7cyclictest31109-21ntpq19:40:241
3103399188,5cyclictest1243-21hwlatdetect21:55:171
3103399182,2cyclictest131rcu_preempt21:49:251
3103399181,6cyclictest0-21swapper/120:30:261
3103399181,13cyclictest4582-21ssh23:45:021
3103399181,12cyclictest3176-21apt-get00:20:131
3103399181,12cyclictest3176-21apt-get00:20:121
3103399181,10cyclictest5876-21latency_hist20:25:011
3102599186,7cyclictest12-21ksoftirqd/000:15:040
3102599183,11cyclictest0-21swapper/020:20:130
3102599182,11cyclictest0-21swapper/020:25:150
3102599182,11cyclictest0-21swapper/020:25:140
3102599182,10cyclictest8091-21munin-run19:20:010
3102599182,10cyclictest0-21swapper/020:00:130
3102599181,7cyclictest30271-21ssh22:10:210
3102599181,7cyclictest15124-21apt-get23:15:030
3102599181,14cyclictest31117-21kthreadcore20:50:190
3102599181,13cyclictest2423-21apt-get20:55:030
31025991810,5cyclictest3152-21hwlatdetect21:20:140
31025991810,5cyclictest16898-21sh21:10:010
3105299172,9cyclictest0-21swapper/320:05:123
3105299172,9cyclictest0-21swapper/319:30:053
3105299172,10cyclictest0-21swapper/320:10:133
31052991712,3cyclictest0-21swapper/320:20:143
31052991711,4cyclictest0-21swapper/320:40:153
3104499171,7cyclictest31067-21latency_hist19:10:012
3104499171,7cyclictest31067-21latency_hist19:10:012
3103399172,10cyclictest0-21swapper/120:35:141
3102599178,7cyclictest28877-21apt-get20:15:140
3102599171,12cyclictest664-21ntpd19:40:130
3102599171,12cyclictest30162-21latency_hist20:50:010
31025991710,5cyclictest0-21swapper/019:40:010
3102599170,13cyclictest19289-21apt-get20:05:100
3105299162,9cyclictest0-21swapper/319:40:133
3103399161,8cyclictest8516-21fschecks_time19:20:161
3103399160,13cyclictest6238-21cut20:25:171
3103399160,13cyclictest6238-21cut20:25:161
3102599162,10cyclictest0-21swapper/020:35:130
3102599161,12cyclictest5010-21cut19:50:150
3102599161,12cyclictest12142-21tr20:30:190
31025991610,3cyclictest32686-21grep19:45:170
3102599160,3cyclictest53646usb-storage19:30:100
3105299159,4cyclictest0-21swapper/319:50:263
3105299151,9cyclictest0-21swapper/320:45:123
3102599152,9cyclictest0-21swapper/020:10:130
3105299149,3cyclictest0-21swapper/319:55:283
3102599141,10cyclictest31087-21latency_hist19:10:020
3102599141,10cyclictest31087-21latency_hist19:10:010
3103399131,9cyclictest31014-21cyclictest19:10:011
3103399131,9cyclictest31014-21cyclictest19:10:011
3105299101,6cyclictest0-21swapper/319:10:023
3105299101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional