You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 13:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Jul 10, 2026 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2892999706,6cyclictest131rcu_preempt21:25:141
2891899564,6cyclictest131rcu_preempt00:25:160
2891899487,28cyclictest28852-21ssh00:00:140
2892999473,35cyclictest16445-21ssh22:05:121
2891899472,40cyclictest5453-21ssh00:05:140
2894599451,34cyclictest0-21swapper/321:40:153
28937994411,14cyclictest11894-21cat22:00:322
2892999444,36cyclictest27212-21rm00:35:131
2891899443,10cyclictest131rcu_preempt23:55:150
2891899421,15cyclictest131rcu_preempt22:05:140
2893799412,27cyclictest27-21ksoftirqd/223:30:122
2891899411,27cyclictest30646-21ssh23:25:150
2894599404,20cyclictest35966irq/58-eth023:45:143
2894599393,26cyclictest35966irq/58-eth020:50:123
2894599393,25cyclictest35966irq/58-eth000:00:133
2893799392,19cyclictest13881-21sh22:39:562
2893799382,28cyclictest15207-21ssh21:26:322
2892999382,33cyclictest12715-21ssh23:15:161
2894599376,25cyclictest35966irq/58-eth000:30:143
28945993717,13cyclictest35966irq/58-eth023:15:153
2893799372,23cyclictest18459-21ssh21:30:112
2894599363,26cyclictest35966irq/58-eth000:15:143
2894599362,27cyclictest35966irq/58-eth021:35:143
2893799363,25cyclictest18089-21ssh23:00:132
2892999362,26cyclictest5309-21ssh22:35:131
2894599355,25cyclictest35966irq/58-eth021:15:173
2894599353,18cyclictest35966irq/58-eth022:20:143
28945993421,7cyclictest35966irq/58-eth023:10:123
2893799333,20cyclictest25169-21ssh23:40:142
28937993313,12cyclictest0-21swapper/200:00:142
2891899332,22cyclictest21671-21ssh23:20:120
2893799323,22cyclictest0-21swapper/200:22:382
2894599314,18cyclictest35966irq/58-eth021:05:133
2893799313,21cyclictest0-21swapper/221:51:322
2891899312,3cyclictest131rcu_preempt23:00:120
2894599301,17cyclictest35966irq/58-eth023:05:143
28945993011,15cyclictest0-21swapper/322:30:143
2893799302,21cyclictest0-21swapper/222:25:162
2893799302,16cyclictest27346irq/40-dwc2_hso19:40:132
2892999303,5cyclictest131rcu_preempt21:40:131
2894599291,17cyclictest35966irq/58-eth019:40:123
2893799293,17cyclictest27346irq/40-dwc2_hso00:10:152
2893799292,20cyclictest0-21swapper/219:30:132
2893799292,19cyclictest27346irq/40-dwc2_hso22:20:142
28937992912,11cyclictest261rcuc/219:20:122
2893799291,13cyclictest3638-21rm23:10:122
28929992922,4cyclictest21974-21ssh21:50:121
2891899291,25cyclictest9527-21ssh21:25:150
2893799286,7cyclictest27-21ksoftirqd/222:41:332
2893799284,18cyclictest27346irq/40-dwc2_hso20:45:112
2893799283,15cyclictest27346irq/40-dwc2_hso21:55:142
2893799282,20cyclictest19396-21ssh23:55:022
28937992810,7cyclictest0-21swapper/223:09:062
2894599273,16cyclictest0-21swapper/322:05:323
2894599273,16cyclictest0-21swapper/322:00:333
2894599273,15cyclictest35966irq/58-eth000:10:133
28945992711,10cyclictest35966irq/58-eth021:27:103
2893799278,12cyclictest27-21ksoftirqd/200:15:162
2893799278,10cyclictest8760-21gpgv22:55:022
2893799277,11cyclictest27-21ksoftirqd/223:28:482
2893799273,17cyclictest0-21swapper/222:05:322
2893799271,20cyclictest25896-21ssh23:57:012
2892999272,20cyclictest20338-21rm21:48:321
2894599263,17cyclictest0-21swapper/321:31:323
2894599263,13cyclictest35966irq/58-eth023:00:133
2893799266,7cyclictest27-21ksoftirqd/200:07:492
2893799264,11cyclictest27346irq/40-dwc2_hso20:20:142
2893799262,18cyclictest0-21swapper/219:15:122
2893799262,14cyclictest27346irq/40-dwc2_hso20:30:122
2893799261,15cyclictest27346irq/40-dwc2_hso20:00:142
2893799261,14cyclictest27346irq/40-dwc2_hso19:50:122
2893799260,4cyclictest271ksoftirqd/200:26:022
2893799260,17cyclictest21479-21ls23:20:022
2892999260,23cyclictest0-21swapper/121:35:121
2891899261,2cyclictest131rcu_preempt22:50:120
2894599254,13cyclictest0-21swapper/323:40:143
2894599254,12cyclictest0-21swapper/323:35:143
2894599254,12cyclictest0-21swapper/323:35:133
2894599253,14cyclictest0-21swapper/322:10:133
28945992513,7cyclictest35966irq/58-eth022:40:133
2893799258,7cyclictest27-21ksoftirqd/221:19:302
2893799254,11cyclictest27-21ksoftirqd/200:34:592
2893799253,14cyclictest27346irq/40-dwc2_hso19:35:122
2893799252,8cyclictest31099-21ssh22:48:562
2893799252,4cyclictest27-21ksoftirqd/221:40:212
2893799252,17cyclictest27346irq/40-dwc2_hso20:50:122
2892999251,19cyclictest9151-21ssh00:25:161
28929992511,9cyclictest2540-21ssh00:02:031
2892999251,19cyclictest23433-21ssh22:45:151
2891899251,18cyclictest27063-21ssh23:05:140
2891899251,17cyclictest23555-21ssh00:15:130
2894599244,16cyclictest35966irq/58-eth023:55:143
2894599243,15cyclictest0-21swapper/322:15:183
2894599243,15cyclictest0-21swapper/300:05:163
2893799246,4cyclictest27-21ksoftirqd/222:31:582
2893799246,10cyclictest27-21ksoftirqd/220:11:592
2893799245,7cyclictest131rcu_preempt21:35:142
2893799245,12cyclictest27-21ksoftirqd/220:40:232
2893799244,8cyclictest27-21ksoftirqd/223:36:562
2893799244,8cyclictest27-21ksoftirqd/223:36:562
2893799243,15cyclictest27-21ksoftirqd/223:48:272
28937992412,9cyclictest0-21swapper/221:45:232
2892999249,8cyclictest131rcu_preempt22:55:151
28929992416,5cyclictest2023-21ssh22:15:181
2891899242,18cyclictest20337-21sh21:48:320
2891899242,17cyclictest14669-21diskmemload21:24:320
28918992414,5cyclictest0-21swapper/021:35:140
2894599233,11cyclictest0-21swapper/323:50:143
28945992313,6cyclictest35966irq/58-eth000:25:163
2893799235,8cyclictest27-21ksoftirqd/200:38:442
2893799234,7cyclictest27-21ksoftirqd/222:56:112
2893799233,8cyclictest27-21ksoftirqd/221:20:052
2893799232,6cyclictest0-21swapper/222:18:522
28929992316,4cyclictest0-21swapper/122:40:141
28929992313,5cyclictest19331-21sh00:10:341
2891899232,15cyclictest10400-21munin-run23:50:010
2891899231,16cyclictest25179-21ssh23:40:150
28945992215,5cyclictest35966irq/58-eth022:55:263
2893799229,4cyclictest27-21ksoftirqd/223:23:322
28937992217,3cyclictest261rcuc/222:13:192
2893799221,11cyclictest16828-21ssh21:10:192
28937992210,9cyclictest18185-21tr20:40:012
28929992213,4cyclictest21-21ksoftirqd/122:00:121
2891899226,11cyclictest12-21ksoftirqd/022:55:040
2893799216,6cyclictest20883-21switchtime20:05:272
2893799215,7cyclictest131rcu_preempt19:45:142
2893799215,6cyclictest131rcu_preempt19:10:172
2893799213,5cyclictest0-21swapper/219:25:132
2893799213,11cyclictest27346irq/40-dwc2_hso19:55:112
2893799212,16cyclictest31194-21usage_EthcTask20:15:292
2893799212,11cyclictest9873-21latency_hist21:05:012
2893799211,16cyclictest10415-21ps21:05:182
2892999214,6cyclictest21-21ksoftirqd/122:50:311
28929992115,4cyclictest1731-21ssh23:45:151
2891899213,11cyclictest12-21ksoftirqd/020:55:120
2894599209,6cyclictest0-21swapper/322:53:263
2894599203,12cyclictest0-21swapper/321:52:333
28945992011,4cyclictest0-21swapper/300:35:153
2894599201,12cyclictest0-21swapper/323:30:133
28945992010,6cyclictest0-21swapper/321:45:313
28945992010,5cyclictest35966irq/58-eth023:20:133
2893799206,5cyclictest6299-21kthreadcore20:25:212
2892999208,8cyclictest21-21ksoftirqd/122:31:561
2892999206,9cyclictest5277-21ssh00:05:041
2892999203,13cyclictest18104-21ssh23:00:141
28929992015,3cyclictest14669-21diskmemload21:15:151
28929992014,3cyclictest0-21swapper/122:10:121
28929992013,4cyclictest30573-21missed_timers23:05:251
28929992013,4cyclictest0-21swapper/120:00:181
2892999201,13cyclictest11372-21timerwakeupswit19:55:271
2891899202,3cyclictest131rcu_preempt21:55:150
2891899202,14cyclictest14363-21ssh22:40:130
2891899201,7cyclictest20201-21irqcore22:25:200
28918992014,4cyclictest0-21swapper/000:24:070
28918992013,5cyclictest0-21swapper/021:00:310
28918992011,7cyclictest30557-21cron21:55:010
2891899201,16cyclictest1546-21cut19:15:140
2891899201,16cyclictest14385-21ls22:20:250
2891899201,16cyclictest12903-21cut23:15:190
2891899201,13cyclictest3636-21ssh23:10:120
2891899200,6cyclictest27224-21ssh00:35:140
2894599193,9cyclictest0-21swapper/322:39:563
2894599193,8cyclictest35966irq/58-eth021:55:133
2894599193,11cyclictest0-21swapper/321:23:323
2894599192,11cyclictest0-21swapper/320:35:133
28945991913,4cyclictest0-21swapper/322:25:143
2894599191,11cyclictest0-21swapper/320:40:213
2893799191,9cyclictest532-21grep20:55:162
2892999199,8cyclictest14449-21ntp_192.168.11422:20:251
2892999198,3cyclictest131rcu_preempt21:10:011
2892999195,7cyclictest18455-21apt-get21:30:121
2892999191,9cyclictest25384-21idleruntime-cro19:40:011
2892999191,14cyclictest32542-21apt-get00:20:131
2892999191,14cyclictest31548-21latency_hist20:20:011
28929991911,4cyclictest0-21swapper/123:35:141
28929991911,4cyclictest0-21swapper/123:35:141
28929991910,6cyclictest12298-21sshd23:15:021
28929991910,6cyclictest0-21swapper/119:10:231
2892999190,4cyclictest131rcu_preempt00:30:331
2891899194,5cyclictest12-21ksoftirqd/022:45:270
2891899191,8cyclictest0-21swapper/020:35:290
28918991913,4cyclictest0-21swapper/019:35:270
28918991911,6cyclictest549-21sshd22:13:520
2891899191,14cyclictest15134-21ls21:10:130
2891899191,13cyclictest28736-21ssh22:30:130
2891899191,13cyclictest26591-21latency_hist20:15:010
2891899191,13cyclictest12619-21cut20:00:160
2891899191,12cyclictest7514-21ls21:40:230
28918991911,2cyclictest14669-21diskmemload00:10:310
28918991910,7cyclictest18923-21irqstats00:30:190
2894599189,7cyclictest33-21ksoftirqd/322:45:183
2894599183,10cyclictest0-21swapper/300:23:073
2894599182,11cyclictest0-21swapper/320:15:133
2894599182,10cyclictest0-21swapper/320:45:123
2894599182,10cyclictest0-21swapper/320:20:133
2894599182,10cyclictest0-21swapper/320:05:133
2894599182,10cyclictest0-21swapper/319:45:143
2894599182,10cyclictest0-21swapper/319:35:043
2894599181,11cyclictest0-21swapper/319:55:253
28945991810,4cyclictest0-21swapper/323:25:173
2893799180,9cyclictest28950-21latency_hist19:10:022
2892999189,7cyclictest8722-21ntp_kernel_pll_21:00:251
2892999188,7cyclictest23122-21sshd21:14:301
2892999187,7cyclictest21-21ksoftirqd/123:40:131
2892999185,11cyclictest30928-21/usr/sbin/munin21:55:241
2892999182,4cyclictest21-21ksoftirqd/121:20:201
2892999182,13cyclictest21684-21sh23:20:131
2892999181,9cyclictest29209-21kthreadcore20:50:201
2892999181,9cyclictest0-21swapper/122:28:041
28929991811,5cyclictest4261-21sshd23:27:011
28929991811,4cyclictest0-21swapper/120:40:141
2892999181,13cyclictest19820-21ssh23:55:151
28929991810,6cyclictest8578-21cron20:30:001
2892999180,9cyclictest2212-21kthreadcore20:20:211
2891899188,8cyclictest1174-21users19:10:320
2891899181,6cyclictest8548-21users20:25:300
2891899181,6cyclictest14669-21diskmemload21:34:420
2891899181,4cyclictest0-21swapper/023:50:210
2891899181,3cyclictest0-21swapper/021:20:010
28918991811,5cyclictest0-21swapper/019:20:160
2891899181,10cyclictest25754-21cut19:40:160
28918991810,6cyclictest12191-21sshd22:37:560
28918991810,5cyclictest1973-21/usr/sbin/munin22:15:320
2894599179,6cyclictest0-21swapper/321:10:303
2894599172,10cyclictest0-21swapper/320:30:133
2894599172,10cyclictest0-21swapper/319:20:143
2894599172,10cyclictest0-21swapper/319:10:153
28945991712,3cyclictest33-21ksoftirqd/321:00:293
2892999179,6cyclictest5990-21cron19:20:011
2892999176,2cyclictest131rcu_preempt00:15:051
2892999172,12cyclictest7138-21apt-get23:30:131
2892999172,12cyclictest10796-21ssh23:50:141
2892999172,11cyclictest6243-21apt-get19:20:111
28929991713,2cyclictest0-21swapper/120:45:121
2892999170,14cyclictest402-21apt-get20:55:111
2891899179,6cyclictest7220-21sh23:30:130
2891899179,6cyclictest30088-21sed20:15:210
2891899179,6cyclictest19069-21cut23:35:230
2891899179,6cyclictest19069-21cut23:35:230
2891899177,8cyclictest26998-21grep20:45:280
2891899175,9cyclictest12658-21unixbench_singl20:30:280
2891899172,9cyclictest7515-21ssh22:00:130
2891899172,13cyclictest13445-21kthreadcore19:25:210
2891899171,6cyclictest20110-21kthreadcore20:40:190
2891899171,6cyclictest16525-21/usr/sbin/munin19:30:190
2892999166,6cyclictest21-21ksoftirqd/120:05:121
2892999165,9cyclictest26384-21kthreadcore19:40:181
2892999165,9cyclictest22443-21irqcore20:10:171
2892999161,8cyclictest14369-21latency19:25:231
28929991611,3cyclictest17099-21processes20:35:261
2892999160,6cyclictest3481-21sed19:50:191
2891899168,6cyclictest2879-21grep20:20:270
28918991611,3cyclictest7844-21idleruntime19:55:140
28918991611,3cyclictest20800-21processes20:05:260
2894599152,9cyclictest0-21swapper/319:30:133
2894599152,8cyclictest35966irq/58-eth019:50:133
2894599152,10cyclictest0-21swapper/320:55:133
2892999152,10cyclictest16078-21fschecks_time19:30:151
2892999151,11cyclictest8813-21apt-get20:30:121
2892999151,11cyclictest30537-21fschecks_count19:45:161
2891899157,6cyclictest30909-21irqrtprio19:45:180
2891899151,11cyclictest3878-21tr19:50:190
2891899151,11cyclictest12667-21kthreadcore21:05:220
2891899151,10cyclictest28961-21cat19:10:020
2891899150,11cyclictest31569-21wc20:50:260
2894599149,3cyclictest0-21swapper/319:25:153
2892999141,11cyclictest28989-21latency_hist19:10:021
2894599133,8cyclictest0-21swapper/319:20:013
2894599131,8cyclictest0-21swapper/320:25:293
2894599111,8cyclictest0-21swapper/320:10:153
2894599111,7cyclictest0-21swapper/320:00:193
2894599101,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional