You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 18:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon May 11, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
173899511,35cyclictest11423-21ssh12:05:141
1757994618,25cyclictest35966irq/58-eth011:00:123
174799462,32cyclictest20360-21ssh12:10:132
174799462,32cyclictest20360-21ssh12:10:132
174799461,24cyclictest22340-21sh10:05:132
172299464,36cyclictest9584-21ssh09:40:130
172299463,32cyclictest4244-21ssh11:25:130
175799443,27cyclictest35966irq/58-eth010:35:133
1738994418,3cyclictest21-21ksoftirqd/107:15:121
1747994215,19cyclictest1708-21cyclictest11:50:122
173899422,19cyclictest25662-21ssh11:55:131
1747994118,18cyclictest1-21systemd08:35:132
174799411,33cyclictest0-21swapper/211:10:142
174799411,33cyclictest0-21swapper/211:10:132
173899413,23cyclictest27635-21ssh11:20:121
172299401,30cyclictest20137-21diskmemload12:35:150
175799391,8cyclictest35966irq/58-eth009:25:133
174799391,27cyclictest29180-21ssh12:15:142
173899392,3cyclictest131rcu_preempt11:35:131
174799387,7cyclictest0-21swapper/209:00:122
173899384,26cyclictest32578-21ssh12:35:131
175799373,12cyclictest35966irq/58-eth012:10:133
175799373,12cyclictest35966irq/58-eth012:10:133
174799377,13cyclictest261rcuc/210:50:132
175799362,20cyclictest35966irq/58-eth011:35:133
174799369,15cyclictest13368-1kworker/u9:211:00:142
174799347,16cyclictest27-21ksoftirqd/210:30:142
174799347,16cyclictest27-21ksoftirqd/210:30:142
174799331,21cyclictest7925-21ssh10:15:132
1747993216,7cyclictest27-21ksoftirqd/209:20:142
172299321,25cyclictest4342-21ssh09:55:140
174799315,16cyclictest27346irq/40-dwc2_hso11:45:122
174799312,17cyclictest5768-21ssh12:20:132
174799311,15cyclictest0-21swapper/211:30:122
172299311,2cyclictest131rcu_preempt09:10:130
175799304,18cyclictest35966irq/58-eth011:30:123
174799302,22cyclictest31375-21ssh10:10:142
174799301,13cyclictest0-21swapper/207:40:142
174799293,17cyclictest20540-21ssh09:10:142
174799293,16cyclictest27346irq/40-dwc2_hso10:40:132
173899291,20cyclictest0-21swapper/108:10:141
175799286,8cyclictest35966irq/58-eth010:45:133
175799285,13cyclictest35966irq/58-eth009:15:133
175799281,15cyclictest35966irq/58-eth009:55:143
175799281,13cyclictest0-21swapper/309:20:233
174799283,14cyclictest27346irq/40-dwc2_hso08:00:122
174799282,18cyclictest22193-21ssh11:35:142
174799282,15cyclictest693-21ssh11:05:152
174799282,15cyclictest14676-21ssh12:25:142
175799279,6cyclictest0-21swapper/311:05:253
175799272,9cyclictest0-21swapper/309:45:133
175799272,10cyclictest0-21swapper/310:40:133
1757992717,7cyclictest35966irq/58-eth009:30:133
1757992710,3cyclictest0-21swapper/311:15:213
174799272,19cyclictest0-21swapper/207:35:132
174799271,21cyclictest21196-21cut07:30:162
175799268,7cyclictest0-21swapper/312:24:543
175799263,5cyclictest0-21swapper/310:05:293
175799262,17cyclictest0-21swapper/310:30:153
175799262,17cyclictest0-21swapper/310:30:153
175799261,13cyclictest35966irq/58-eth008:20:123
174799262,6cyclictest27-21ksoftirqd/209:45:132
174799261,21cyclictest5843-21ls08:55:012
174799261,18cyclictest0-21swapper/209:05:142
1747992610,12cyclictest10481-21grep07:15:282
172299261,20cyclictest27648-21apt-get09:50:140
172299261,19cyclictest19056-21sd-accept08:35:130
175799259,4cyclictest0-21swapper/311:20:203
174799257,15cyclictest31744-21kthreadcore09:15:212
174799255,13cyclictest6775-21cut07:45:272
174799252,14cyclictest27346irq/40-dwc2_hso10:20:132
174799251,19cyclictest7510-21cut09:55:232
174799251,18cyclictest647-21ssh09:35:132
174799251,18cyclictest647-21ssh09:35:132
1747992510,12cyclictest17358-21cron08:00:002
173899255,12cyclictest20546-21ssh09:10:141
175799242,5cyclictest0-21swapper/312:30:143
174799244,6cyclictest13368-1kworker/u9:210:55:142
174799243,17cyclictest0-21swapper/209:25:142
174799242,16cyclictest0-21swapper/210:35:142
174799242,16cyclictest0-21swapper/209:50:052
174799241,19cyclictest32369-21ls12:35:022
174799241,13cyclictest2452-21sh11:40:262
174799241,13cyclictest18676-21ssh11:15:152
174799241,13cyclictest13362-21ssh10:00:132
174799240,21cyclictest28653-21find08:45:122
174799240,12cyclictest27933-21ls11:20:172
1738992415,4cyclictest0-21swapper/111:50:131
1738992411,7cyclictest21-21ksoftirqd/107:30:131
172299243,12cyclictest18587-21ssh09:45:110
172299241,5cyclictest9471-21ps12:20:270
172299241,4cyclictest0-21swapper/011:30:020
175799239,9cyclictest35966irq/58-eth012:05:143
1757992312,6cyclictest35966irq/58-eth009:50:153
174799231,19cyclictest2805-21tr12:35:232
174799231,11cyclictest25681-21sh11:55:132
1738992311,3cyclictest20137-21diskmemload12:13:511
1738992311,3cyclictest20137-21diskmemload12:13:511
172299231,18cyclictest29133-21find12:15:110
172299231,17cyclictest2423-21ssh12:00:150
172299231,15cyclictest25296-21kworker/u8:212:30:140
1757992214,2cyclictest0-21swapper/311:25:133
174799228,11cyclictest19672-21netstat07:25:232
174799223,15cyclictest4119-21kthreadcore12:00:222
1747992214,5cyclictest536-21usb-storage11:30:022
174799221,12cyclictest27284-21kthreadcore10:25:212
174799221,11cyclictest28047-21ls09:30:292
173899227,10cyclictest21-21ksoftirqd/112:00:141
173899223,6cyclictest21-21ksoftirqd/107:40:121
173899222,12cyclictest29554-21ssh10:45:131
1738992212,7cyclictest5342-21cron12:20:011
1722992212,7cyclictest17368-21munin-plugin-st08:00:010
174799219,9cyclictest654-21ssh10:45:242
174799212,8cyclictest7371-21kthreadcore08:20:212
174799212,12cyclictest9569-21apt-get09:40:112
174799212,11cyclictest8849-21kthreadcore08:55:222
174799211,11cyclictest3047-21missed_timers08:15:212
174799211,11cyclictest17981-21grep08:30:302
174799211,11cyclictest11868-21sed07:50:292
174799211,11cyclictest0-21swapper/207:10:192
1747992110,8cyclictest11384-21/usr/sbin/munin12:05:182
174799210,11cyclictest26955-21kthreadcore08:40:222
173899212,16cyclictest16970-21apt-get10:20:131
173899212,14cyclictest18569-21apt-get09:45:111
173899211,15cyclictest9791-21ntp_192.168.11410:50:261
172299216,3cyclictest131rcu_preempt11:00:000
172299211,7cyclictest752-21kthreadcore10:10:220
1722992112,7cyclictest1-21systemd10:25:390
172299211,17cyclictest16637-21cut10:00:230
175799209,8cyclictest0-21swapper/309:11:383
1757992011,6cyclictest0-21swapper/310:00:163
175799200,11cyclictest35966irq/58-eth008:30:133
174799201,9cyclictest27164-21grep08:10:022
174799201,16cyclictest27974-21ps08:10:202
174799200,10cyclictest15282-21tail07:20:282
173899207,2cyclictest131rcu_preempt09:00:121
173899205,8cyclictest0-21swapper/107:25:121
173899204,5cyclictest21-21ksoftirqd/111:25:301
1738992012,6cyclictest0-21swapper/108:30:181
1738992010,7cyclictest12134-21cat10:15:291
172299207,3cyclictest20137-21diskmemload11:59:020
172299202,14cyclictest9700-21ssh11:10:130
172299202,14cyclictest9700-21ssh11:10:130
172299201,9cyclictest0-21swapper/011:05:520
1722992011,6cyclictest22754-21tr11:35:190
172299201,15cyclictest20607-21ssh10:40:130
172299201,11cyclictest20137-21diskmemload11:01:560
172299201,10cyclictest1689766kworker/u9:110:35:230
175799198,8cyclictest0-21swapper/310:59:353
175799191,13cyclictest0-21swapper/310:10:153
175799191,12cyclictest0-21swapper/308:50:143
175799191,12cyclictest0-21swapper/308:00:173
175799191,11cyclictest0-21swapper/307:55:013
1757991910,6cyclictest0-21swapper/311:45:213
1757991910,6cyclictest0-21swapper/310:20:153
174799192,13cyclictest1771-21grep07:10:022
174799191,8cyclictest9479-21fschecks_count08:25:162
173899199,6cyclictest21-21ksoftirqd/111:15:141
173899193,11cyclictest0-21swapper/108:35:131
173899191,6cyclictest1999-21kthreadcore09:35:191
173899191,6cyclictest1999-21kthreadcore09:35:191
1738991914,3cyclictest0-21swapper/108:15:121
1738991912,4cyclictest0-21swapper/111:30:121
1738991912,4cyclictest0-21swapper/110:30:141
1738991912,4cyclictest0-21swapper/110:30:141
173899191,13cyclictest11696-21ssh10:35:161
1738991910,6cyclictest0-21swapper/109:25:241
173899190,16cyclictest15544-21ssh12:25:201
173899190,14cyclictest9626-21ls08:55:281
172299198,8cyclictest31452-21ps09:15:200
172299197,9cyclictest26437-21grep10:05:290
172299197,9cyclictest12547-21cron07:55:010
172299192,13cyclictest29573-21diskmemload10:45:140
172299191,9cyclictest0-21swapper/012:25:500
172299191,9cyclictest0-21swapper/012:12:530
172299191,9cyclictest0-21swapper/012:12:530
172299191,6cyclictest9782-21ntp_192.168.11410:50:250
172299191,2cyclictest0-21swapper/009:35:210
172299191,2cyclictest0-21swapper/009:35:210
172299191,16cyclictest29180-21grep08:10:210
1722991911,6cyclictest18753-21http08:35:020
1722991911,6cyclictest12035-21sh10:15:290
172299191,14cyclictest11422-21sh12:05:140
172299191,12cyclictest0-21swapper/007:20:140
172299191,10cyclictest0-21swapper/011:52:410
1722991910,7cyclictest21591-21kthreadcore11:15:240
1722991910,6cyclictest17846-21wc08:00:170
175799189,6cyclictest35966irq/58-eth011:55:143
175799189,6cyclictest0-21swapper/311:40:173
175799187,7cyclictest35966irq/58-eth010:15:123
175799182,9cyclictest0-21swapper/309:05:123
175799182,10cyclictest0-21swapper/309:00:133
175799182,10cyclictest0-21swapper/308:55:133
175799182,10cyclictest0-21swapper/308:35:133
175799182,10cyclictest0-21swapper/307:45:043
175799182,10cyclictest0-21swapper/307:30:143
175799182,10cyclictest0-21swapper/307:15:143
1757991811,5cyclictest35966irq/58-eth012:00:303
175799181,13cyclictest0-21swapper/307:35:263
1757991810,5cyclictest35966irq/58-eth012:25:143
1757991810,5cyclictest35966irq/58-eth011:50:123
173899189,7cyclictest14769-21memory07:20:231
173899188,7cyclictest3413-21cat10:10:301
173899186,3cyclictest131rcu_preempt11:41:011
173899185,10cyclictest2192-21ps07:10:181
173899183,12cyclictest6470-21irqcore09:20:181
173899183,10cyclictest0-21swapper/107:50:131
173899182,12cyclictest24165-21ssh09:30:141
1738991812,3cyclictest0-21swapper/109:50:151
173899181,14cyclictest29469-21apt-get09:15:121
1738991811,4cyclictest20137-21diskmemload11:00:131
173899181,13cyclictest7770-21ssh11:45:141
173899181,12cyclictest7975-21ssh09:55:261
1738991810,5cyclictest28800-21sort08:45:151
172299188,8cyclictest15547-21apt-get09:05:130
172299188,7cyclictest27925-21wc11:20:170
172299185,7cyclictest0-21swapper/011:40:130
172299181,15cyclictest32285-21cut08:15:140
172299181,13cyclictest11696-21awk11:45:290
172299181,12cyclictest0-21swapper/008:45:140
1722991810,6cyclictest9578-21sh09:20:240
175799179,6cyclictest0-21swapper/312:35:033
175799179,6cyclictest0-21swapper/312:16:143
175799179,6cyclictest0-21swapper/310:50:043
175799179,6cyclictest0-21swapper/309:35:133
175799179,6cyclictest0-21swapper/309:35:133
175799172,10cyclictest0-21swapper/311:10:143
175799172,10cyclictest0-21swapper/311:10:143
175799172,10cyclictest0-21swapper/310:25:153
175799172,10cyclictest0-21swapper/308:15:123
175799171,13cyclictest0-21swapper/308:45:293
175799171,12cyclictest0-21swapper/308:40:203
173899179,6cyclictest21507-21cut10:40:191
173899172,12cyclictest25983-21ssh10:25:151
173899171,7cyclictest9858-21sed08:25:181
173899171,7cyclictest15869-21irqstats10:55:201
1738991713,2cyclictest0-21swapper/108:50:111
173899171,14cyclictest26130-21wc10:05:281
173899171,13cyclictest9582-21sh09:40:131
173899171,12cyclictest24856-21tr12:30:211
173899171,12cyclictest12990-21grep07:55:171
173899171,10cyclictest9737-21ls12:20:281
1738991710,4cyclictest13746-21irqcore10:00:191
173899170,13cyclictest26000-21uniq07:35:151
172299179,5cyclictest29641-21cat07:35:270
172299179,5cyclictest12752-21ntp_kernel_err08:25:250
172299179,5cyclictest0-21swapper/007:15:250
172299178,7cyclictest9115-21cron08:25:010
172299176,8cyclictest2099-21grep07:40:300
172299172,2cyclictest131rcu_preempt08:55:020
172299172,12cyclictest24040-21apt-get09:30:130
172299171,8cyclictest0-21swapper/009:28:320
172299171,4cyclictest20137-21diskmemload10:32:120
172299171,4cyclictest20137-21diskmemload10:32:120
172299171,10cyclictest844-21munin-node10:20:140
175799162,10cyclictest0-21swapper/307:55:143
175799162,10cyclictest0-21swapper/307:40:143
175799162,10cyclictest0-21swapper/307:10:143
175799161,11cyclictest0-21swapper/309:40:213
173899169,4cyclictest9126-1kworker/u9:211:10:141
173899169,4cyclictest9126-1kworker/u9:211:10:141
173899167,6cyclictest1019-21cat11:05:191
173899162,10cyclictest22443-21/usr/sbin/munin08:05:121
173899161,6cyclictest9124-21munin-run08:25:011
173899160,13cyclictest17602-21/usr/sbin/munin08:00:171
172299169,5cyclictest25632-21cron07:35:010
172299169,5cyclictest16600-21irqcore07:25:180
172299163,10cyclictest4201-21kthreadcore07:45:190
172299161,12cyclictest10793-21apt-get09:00:130
172299161,11cyclictest0-21swapper/007:10:140
175799151,8cyclictest0-21swapper/308:05:163
175799151,8cyclictest0-21swapper/307:25:133
175799151,7cyclictest0-21swapper/308:30:013
173899159,4cyclictest16442-21kthreadcore09:05:191
173899157,6cyclictest7771-21cron07:50:011
173899152,10cyclictest24165-21if_eth108:40:171
172299157,6cyclictest25170-21kthreadcore08:05:220
172299152,10cyclictest26635-21kthreadcore08:40:200
172299150,5cyclictest53646usb-storage08:55:110
172299150,10cyclictest1621-21latency_hist07:10:020
175799149,3cyclictest0-21swapper/307:20:173
175799141,10cyclictest0-21swapper/308:10:143
173899131,9cyclictest1811-21latency_hist07:10:011
175799101,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional