You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-29 - 12:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Tue Sep 29, 2026 00:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2422399735,56cyclictest25868-21ssh00:00:160
2422399635,37cyclictest17760-21ssh21:50:120
2422399586,46cyclictest30708-21ssh22:15:120
24257995644,7cyclictest35966irq/58-eth023:30:133
24257995644,7cyclictest35966irq/58-eth023:30:133
2423599552,36cyclictest2458-21ssh00:05:141
2422399557,39cyclictest18479-21ssh23:20:130
2422399533,33cyclictest32098-21ssh21:40:120
2422399521,37cyclictest12624-21ssh22:05:130
24247994720,20cyclictest27-21ksoftirqd/223:00:122
2424799463,29cyclictest5675-21ssh22:55:102
2425799446,24cyclictest35966irq/58-eth020:20:133
2422399442,30cyclictest29562-21ssh00:20:120
24257994325,12cyclictest35966irq/58-eth022:20:143
2425799424,23cyclictest35966irq/58-eth000:15:163
24257994227,12cyclictest35966irq/58-eth000:10:143
2425799412,24cyclictest35966irq/58-eth000:05:133
2423599413,27cyclictest5684-21ssh22:55:121
2424799406,25cyclictest6245-21ssh00:25:132
24223994032,5cyclictest121ksoftirqd/022:20:140
2422399399,22cyclictest12-21ksoftirqd/022:45:140
2422399399,22cyclictest12-21ksoftirqd/022:45:140
2422399392,4cyclictest131rcu_preempt21:30:120
2422399391,8cyclictest131rcu_preempt23:45:120
24223993917,13cyclictest12-21ksoftirqd/020:55:130
24223993917,13cyclictest12-21ksoftirqd/020:55:130
2423599382,26cyclictest6236-21ssh00:25:131
2422399388,25cyclictest9904-21diskmemload22:40:140
2422399384,22cyclictest12-21ksoftirqd/022:35:130
2422399378,20cyclictest12-21ksoftirqd/021:00:140
2422399371,30cyclictest11575-21ssh00:10:140
2425799362,23cyclictest35966irq/58-eth022:10:143
2424799361,16cyclictest261rcuc/221:45:112
2425799352,20cyclictest35966irq/58-eth021:25:133
2425799343,18cyclictest35966irq/58-eth023:35:133
24257993424,8cyclictest35966irq/58-eth022:50:163
2425799341,18cyclictest35966irq/58-eth023:15:133
2425799334,14cyclictest0-21swapper/323:45:133
2425799332,17cyclictest0-21swapper/300:00:163
2422399332,21cyclictest4963-21ssh21:25:140
2425799322,18cyclictest35966irq/58-eth019:20:133
2423599323,5cyclictest21-21ksoftirqd/120:20:251
24235993218,9cyclictest2903-21apt-get19:55:131
24235993218,9cyclictest2903-21apt-get19:55:121
2422399321,23cyclictest0-21swapper/000:30:130
24257993117,11cyclictest35966irq/58-eth021:50:113
2422399313,22cyclictest22240-21ssh23:40:170
2422399312,20cyclictest20608-21ssh00:15:150
2422399311,23cyclictest14700-21ssh23:00:130
2424799305,20cyclictest3664-21cut00:05:212
2424799304,16cyclictest27-21ksoftirqd/200:20:142
2424799302,20cyclictest16791-21ssh23:55:132
24247993013,13cyclictest0-21swapper/223:50:112
2424799301,23cyclictest31220-21ssh23:45:122
2424799301,10cyclictest28314-21find21:20:122
2423599302,22cyclictest16291-21ssh22:25:111
2424799293,21cyclictest0-21swapper/220:10:142
24247992916,9cyclictest21516-21sh21:50:262
24235992915,7cyclictest28155-21apt-get20:55:131
24235992915,7cyclictest28155-21apt-get20:55:131
24235992915,11cyclictest21-21ksoftirqd/121:20:131
24223992918,7cyclictest0-21swapper/021:20:140
2425799283,18cyclictest35966irq/58-eth020:45:143
2425799282,18cyclictest35966irq/58-eth021:20:143
2424799282,21cyclictest22439-21ssh00:33:112
2424799281,15cyclictest26875-21ssh21:55:132
2423599282,21cyclictest25438-21ssh22:30:191
2422399284,13cyclictest24422-21ssh00:35:150
2422399281,4cyclictest131rcu_preempt20:45:140
2422399280,21cyclictest26859-21ssh21:55:120
2425799273,16cyclictest0-21swapper/300:35:213
2425799273,14cyclictest35966irq/58-eth021:35:113
2424799273,9cyclictest0-21swapper/223:30:142
2424799273,9cyclictest0-21swapper/223:30:142
2424799273,19cyclictest261rcuc/222:45:132
2424799273,19cyclictest261rcuc/222:45:132
2424799270,17cyclictest32521-21ls23:10:022
2424799270,17cyclictest0-21swapper/219:15:122
2422399272,18cyclictest3662-21kthreadcore00:05:200
24223992716,5cyclictest12-21ksoftirqd/023:50:110
2422399271,2cyclictest131rcu_preempt20:15:130
2425799262,20cyclictest35966irq/58-eth020:35:133
24257992610,10cyclictest35966irq/58-eth023:05:153
2424799265,10cyclictest27346irq/40-dwc2_hso19:35:132
2424799265,10cyclictest27346irq/40-dwc2_hso19:35:132
2424799262,17cyclictest27346irq/40-dwc2_hso20:55:132
2424799262,17cyclictest27346irq/40-dwc2_hso20:55:132
24247992610,7cyclictest1586-21cron22:35:012
2423599268,12cyclictest1974-21apt-get22:35:141
2425799254,10cyclictest0-21swapper/321:45:133
24257992512,10cyclictest35966irq/58-eth022:55:123
2425799251,14cyclictest0-21swapper/321:55:123
24257992510,10cyclictest35966irq/58-eth021:40:133
2424799254,14cyclictest27346irq/40-dwc2_hso22:00:112
2424799252,18cyclictest27346irq/40-dwc2_hso21:25:122
2424799251,15cyclictest930-21snmpd19:50:142
2424799251,13cyclictest20593-21ssh00:15:152
2423599257,9cyclictest21-21ksoftirqd/123:55:431
2423599257,9cyclictest21-21ksoftirqd/121:52:441
2423599257,8cyclictest21-21ksoftirqd/123:37:581
2423599256,9cyclictest21-21ksoftirqd/121:33:311
2423599256,9cyclictest21-21ksoftirqd/100:37:581
2423599253,4cyclictest21-21ksoftirqd/122:00:221
2423599252,19cyclictest7303-21apt-get22:20:121
24235992519,4cyclictest201rcuc/121:18:211
2422399255,14cyclictest151rcuc/021:45:120
24223992515,7cyclictest26769-21kworker/0:100:25:120
2425799248,11cyclictest35966irq/58-eth023:55:123
2425799242,14cyclictest0-21swapper/321:10:163
2424799247,7cyclictest27-21ksoftirqd/221:10:282
2424799242,16cyclictest27346irq/40-dwc2_hso19:45:112
24247992414,8cyclictest0-21swapper/221:40:262
24247992413,9cyclictest1945-21ssh22:35:112
2423599248,8cyclictest21-21ksoftirqd/122:49:481
2423599248,8cyclictest21-21ksoftirqd/122:49:471
2423599248,7cyclictest21-21ksoftirqd/100:33:441
2423599247,9cyclictest21-21ksoftirqd/123:45:061
2423599247,9cyclictest21-21ksoftirqd/122:10:031
2423599247,8cyclictest21-21ksoftirqd/123:33:021
2423599247,8cyclictest21-21ksoftirqd/123:33:011
2423599247,8cyclictest21-21ksoftirqd/121:40:581
2423599247,8cyclictest21-21ksoftirqd/121:11:481
2423599247,7cyclictest21-21ksoftirqd/121:58:001
2423599247,3cyclictest131rcu_preempt22:40:311
2423599246,9cyclictest21-21ksoftirqd/123:20:561
2423599246,8cyclictest21-21ksoftirqd/100:11:171
2423599245,9cyclictest21-21ksoftirqd/100:05:001
2423599243,15cyclictest9382-21ssh23:15:141
2423599243,10cyclictest21-21ksoftirqd/122:15:541
2423599240,6cyclictest28776-21cut19:45:221
2422399242,17cyclictest21663-21ssh22:10:150
2425799233,16cyclictest35966irq/58-eth022:30:183
2424799238,11cyclictest0-21swapper/221:30:122
2424799235,13cyclictest27-21ksoftirqd/223:10:142
2424799232,6cyclictest0-21swapper/200:03:022
24247992317,4cyclictest0-21swapper/223:20:202
24247992312,8cyclictest27-21ksoftirqd/223:15:312
2424799231,11cyclictest9904-21diskmemload22:20:152
2423599237,7cyclictest21-21ksoftirqd/121:45:541
2423599236,9cyclictest21-21ksoftirqd/123:01:461
2423599236,5cyclictest21-21ksoftirqd/123:14:361
2423599235,8cyclictest21-21ksoftirqd/121:29:381
2423599235,6cyclictest131rcu_preempt23:50:171
2423599233,5cyclictest21-21ksoftirqd/119:25:201
24235992312,3cyclictest0-21swapper/123:40:061
2423599231,18cyclictest29571-21ssh00:20:131
2423599231,17cyclictest0-21swapper/120:10:131
24235992310,3cyclictest131rcu_preempt00:18:061
2422399232,14cyclictest29104-21ssh22:50:160
2425799226,9cyclictest35966irq/58-eth000:30:143
24257992212,4cyclictest0-21swapper/321:15:133
2424799224,12cyclictest27346irq/40-dwc2_hso20:30:142
2424799224,12cyclictest27346irq/40-dwc2_hso20:30:132
24247992215,4cyclictest0-21swapper/200:35:152
24247992212,8cyclictest0-21swapper/222:15:162
24247992211,9cyclictest27215-21ssh22:11:212
24247992211,9cyclictest0-21swapper/222:41:292
2423599227,8cyclictest21-21ksoftirqd/123:25:351
2423599227,8cyclictest21-21ksoftirqd/122:08:101
2423599223,4cyclictest21-21ksoftirqd/123:05:221
2423599221,17cyclictest18755-21cut20:45:181
2422399222,12cyclictest4145-21ssh23:30:130
2422399222,12cyclictest4145-21ssh23:30:120
2422399221,16cyclictest16012-21apt-get19:35:130
2422399221,16cyclictest16012-21apt-get19:35:120
2425799219,8cyclictest35966irq/58-eth023:00:133
24257992112,7cyclictest0-21swapper/300:24:203
2425799211,15cyclictest35966irq/58-eth000:25:133
2424799219,4cyclictest27-21ksoftirqd/220:20:132
2424799218,10cyclictest32226-21kthreadcore22:50:262
2424799216,6cyclictest0-21swapper/223:40:002
2424799213,15cyclictest2867-21kthreadcore19:20:202
2424799212,15cyclictest5503-21fschecks_time21:05:182
2424799212,14cyclictest27346irq/40-dwc2_hso19:55:142
2424799212,14cyclictest27346irq/40-dwc2_hso19:55:132
24247992112,7cyclictest0-21swapper/221:19:532
24247992111,8cyclictest0-21swapper/220:15:232
2424799211,16cyclictest27479-21/usr/sbin/munin23:25:142
2424799211,16cyclictest22237-21sh23:40:172
24247992111,4cyclictest0-21swapper/200:10:112
2424799211,11cyclictest20214-21threads22:25:262
2424799211,10cyclictest24922-21kthreadcore21:35:192
2424799210,7cyclictest3062-21timerandwakeup20:25:352
2423599215,6cyclictest21-21ksoftirqd/122:50:371
2423599211,7cyclictest2785-21/usr/sbin/munin20:25:251
24235992115,4cyclictest201rcuc/121:36:321
2423599211,3cyclictest21-21ksoftirqd/119:15:231
2422399212,10cyclictest13194-21ssh23:35:130
2422399211,16cyclictest27586-21ssh22:30:270
2422399211,14cyclictest16308-21sh22:25:120
2425799204,10cyclictest35966irq/58-eth022:40:163
2425799202,11cyclictest0-21swapper/321:00:153
24257992011,6cyclictest0-21swapper/322:28:283
2425799201,13cyclictest0-21swapper/322:05:143
2424799203,11cyclictest0-21swapper/220:40:142
2424799202,9cyclictest11354-21fschecks_count19:30:162
2424799202,10cyclictest13170-21grep22:05:192
2424799201,9cyclictest3445-21kthreadcore21:00:232
2424799201,9cyclictest26707-21kthreadcore19:10:212
24247992011,6cyclictest0-21swapper/220:45:202
2424799201,10cyclictest135-21systemd-journal20:10:012
2424799201,10cyclictest135-21systemd-journal20:10:002
2424799200,6cyclictest27118-21timerwakeupswit20:50:272
2423599209,7cyclictest492-21apt-get21:00:151
2423599206,8cyclictest21-21ksoftirqd/119:40:031
2423599203,4cyclictest21-21ksoftirqd/120:35:291
24235992014,4cyclictest21-21ksoftirqd/119:30:121
24235992011,6cyclictest9896-21cron21:10:021
24235992010,2cyclictest131rcu_preempt19:10:271
2422399202,7cyclictest11061-21irqstats21:10:220
2422399202,15cyclictest9498-21ssh23:15:160
2422399201,7cyclictest9086-21cat23:15:010
2422399201,6cyclictest29097-21sed19:15:010
2422399201,12cyclictest32045-21ls20:25:170
2425799197,9cyclictest0-21swapper/323:25:203
2425799193,10cyclictest35966irq/58-eth020:15:123
2425799191,13cyclictest0-21swapper/320:30:233
2425799191,13cyclictest0-21swapper/320:30:233
24257991910,6cyclictest0-21swapper/322:03:483
24257991910,4cyclictest0-21swapper/323:23:553
2424799192,14cyclictest7950-21fschecks_time20:00:182
2424799191,9cyclictest6605-21grep19:25:172
2424799191,9cyclictest21770-21cut19:40:202
2424799191,14cyclictest9112-21fschecks_count20:35:162
2423599193,3cyclictest21-21ksoftirqd/119:35:261
2423599193,3cyclictest21-21ksoftirqd/119:35:261
2423599193,10cyclictest21-21ksoftirqd/119:20:181
2423599192,5cyclictest21-21ksoftirqd/119:10:021
2423599192,4cyclictest21-21ksoftirqd/120:40:251
2422399195,4cyclictest131rcu_preempt22:00:160
2422399192,9cyclictest3572-21kthreadcore19:55:200
2422399192,9cyclictest3572-21kthreadcore19:55:200
2422399192,13cyclictest13521-21munin-run20:40:010
2422399191,15cyclictest32364-21kthreadcore19:50:210
2422399191,11cyclictest0-21swapper/019:30:140
2422399191,10cyclictest28578-21kthreadcore19:45:210
2425799189,6cyclictest35966irq/58-eth022:45:143
2425799189,6cyclictest35966irq/58-eth022:45:143
2425799189,6cyclictest0-21swapper/321:30:253
2425799182,10cyclictest0-21swapper/319:55:153
2425799182,10cyclictest0-21swapper/319:55:153
2425799182,10cyclictest0-21swapper/319:50:153
24257991810,6cyclictest0-21swapper/323:54:543
24257991810,6cyclictest0-21swapper/323:12:263
24257991810,3cyclictest0-21swapper/320:00:133
2424799181,8cyclictest24278-21grep19:10:012
2423599184,8cyclictest21-21ksoftirqd/119:50:261
2423599183,9cyclictest0-21swapper/120:15:131
2423599183,4cyclictest21-21ksoftirqd/120:00:201
24235991813,3cyclictest0-21swapper/120:05:231
24235991813,3cyclictest0-21swapper/120:05:221
2423599181,12cyclictest0-21swapper/120:50:111
2422399185,3cyclictest131rcu_preempt22:59:150
2422399185,3cyclictest131rcu_preempt20:20:230
2422399182,12cyclictest23029-21ssh21:35:130
24223991812,4cyclictest21241-21unixbench_singl20:10:290
24223991812,4cyclictest0-21swapper/023:25:120
24223991811,4cyclictest0-21swapper/023:05:270
24223991810,5cyclictest12360-21cron20:05:000
24223991810,5cyclictest0-21swapper/021:20:000
2425799179,6cyclictest0-21swapper/323:40:033
2425799179,6cyclictest0-21swapper/322:36:403
2425799179,6cyclictest0-21swapper/322:17:173
2425799179,6cyclictest0-21swapper/320:55:163
2425799179,6cyclictest0-21swapper/320:55:153
2425799172,12cyclictest0-21swapper/320:50:123
2425799171,10cyclictest35966irq/58-eth019:40:133
2422399178,7cyclictest13853-21df_inode20:40:150
2422399171,4cyclictest53646usb-storage19:40:040
2422399170,6cyclictest29839-21/usr/sbin/munin19:15:180
2425799168,6cyclictest0-21swapper/319:39:213
2425799168,6cyclictest0-21swapper/319:39:213
2425799165,7cyclictest35966irq/58-eth019:45:133
2425799162,10cyclictest0-21swapper/319:25:143
2423599161,12cyclictest4251-21diskstats20:30:161
2423599161,12cyclictest4251-21diskstats20:30:151
2422399169,4cyclictest4785-21irqrtprio20:30:190
2422399169,4cyclictest4785-21irqrtprio20:30:190
2422399168,5cyclictest9284-21kthreadcore19:25:220
2422399162,11cyclictest16448-21grep20:05:290
2422399162,11cyclictest16448-21grep20:05:290
2422399161,6cyclictest16769-21/usr/sbin/munin23:55:110
2422399161,12cyclictest2671-21kthreadcore19:20:180
2422399161,12cyclictest23664-21irqcore20:50:170
2422399160,10cyclictest5576-21ls21:05:180
2422399152,9cyclictest24346-21latency_hist19:10:020
2425799141,10cyclictest0-21swapper/321:10:023
2425799138,3cyclictest0-21swapper/319:16:363
2425799131,6cyclictest0-21swapper/320:40:233
2425799131,6cyclictest0-21swapper/319:35:003
2425799131,10cyclictest0-21swapper/320:25:213
2425799125,5cyclictest0-21swapper/319:10:273
2425799122,6cyclictest0-21swapper/320:10:143
2425799122,6cyclictest0-21swapper/320:05:143
2425799122,6cyclictest0-21swapper/320:05:143
2425799111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional