You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 17:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Jul 04, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2035299654,48cyclictest5392-21ssh09:45:141
2035299653,47cyclictest6498-21ssh09:10:141
2035299651,50cyclictest1951-21ssh11:30:141
2036099611,7cyclictest0-21swapper/207:50:122
20360995954,3cyclictest27-21ksoftirqd/211:20:142
2034199567,41cyclictest27635-21ssh10:15:130
20368995335,8cyclictest35966irq/58-eth010:25:153
2034199534,4cyclictest131rcu_preempt11:10:130
2034199534,4cyclictest131rcu_preempt11:10:130
20341995238,7cyclictest0-21swapper/011:20:150
2034199523,35cyclictest15605-21ssh09:15:120
2036899514,27cyclictest35966irq/58-eth011:55:153
2036099514,12cyclictest27346irq/40-dwc2_hso07:10:142
2036099512,28cyclictest1498-21ssh09:25:142
2034199515,36cyclictest14885-21ssh11:55:140
2034199513,36cyclictest21496-21ssh11:05:150
2034199512,10cyclictest131rcu_preempt11:25:140
2036099501,2cyclictest0-21swapper/211:10:132
2036099501,2cyclictest0-21swapper/211:10:132
2034199506,34cyclictest17574-21rm10:45:140
20352994922,23cyclictest21-21ksoftirqd/109:55:131
2035299481,38cyclictest7065-21ssh11:15:131
2035299459,29cyclictest10697-21kworker/u8:012:00:131
2036099433,12cyclictest27346irq/40-dwc2_hso07:30:122
20341994322,10cyclictest12-21ksoftirqd/011:35:130
2036099421,33cyclictest3274-21rm10:55:142
2036099419,5cyclictest27129-21cut11:25:202
2036099414,28cyclictest7074-21ssh11:15:132
20368994018,12cyclictest35966irq/58-eth011:15:133
2036099402,10cyclictest19794-21kthreadcore12:15:192
2036099401,16cyclictest3904-21apt-get12:25:022
2036099400,10cyclictest7103-21ssh11:50:192
2035299403,24cyclictest8500-21ssh10:40:141
2035299403,24cyclictest8500-21ssh10:40:141
20368993922,9cyclictest35966irq/58-eth012:05:263
2036099392,9cyclictest25059-21fschecks_time09:20:162
2036099392,8cyclictest27346irq/40-dwc2_hso11:02:442
20341993916,11cyclictest12-21ksoftirqd/012:15:120
2034199391,32cyclictest13519-21ssh10:25:160
2036899384,20cyclictest35966irq/58-eth011:05:143
2036899382,22cyclictest35966irq/58-eth011:25:143
2036099381,13cyclictest32219-21irqcore10:35:202
2036099381,12cyclictest759-21fschecks_time12:05:162
2036099381,11cyclictest27346irq/40-dwc2_hso09:55:142
20352993812,9cyclictest15370-21sh10:06:441
2036899371,23cyclictest0-21swapper/310:15:133
2036099371,9cyclictest25542-21grep11:05:282
2036099371,9cyclictest11022-21/usr/sbin/munin11:35:272
2036099371,7cyclictest23710-21kthreadcore07:45:212
20360993714,17cyclictest73-21kswapd012:10:122
20360993713,3cyclictest19478-21ssh10:10:192
2036099371,13cyclictest13732-21fschecks_time10:25:192
2036099366,9cyclictest0-21swapper/210:30:182
20360993627,6cyclictest27-21ksoftirqd/210:05:132
2036099361,9cyclictest7786-21kthreadcore08:35:212
2036099361,4cyclictest9422-21sh09:45:282
2036099361,16cyclictest14037-21ls09:30:232
2034199363,24cyclictest12-21ksoftirqd/011:40:130
2034199362,25cyclictest0-21swapper/009:25:140
2036899357,16cyclictest0-21swapper/309:25:143
2036899353,22cyclictest35966irq/58-eth010:55:133
2036099351,5cyclictest12298-21unixbench_multi08:05:282
2036099351,4cyclictest135-21systemd-journal08:50:252
2036099351,12cyclictest4286-21/usr/sbin/munin10:20:172
2036099351,12cyclictest20228-21fschecks_count11:40:162
2036099351,12cyclictest19912-21/usr/sbin/munin09:35:162
20341993521,9cyclictest18563-21sh10:10:120
2036099341,9cyclictest15652-21ssh09:50:182
2036099341,7cyclictest27697-21processes12:00:272
2036099341,10cyclictest15028-21kthreadcore12:30:202
2034199346,20cyclictest151rcuc/012:25:140
20341993420,10cyclictest0-21swapper/012:20:130
20341993411,17cyclictest12-21ksoftirqd/010:35:140
2036899331,17cyclictest35966irq/58-eth012:15:123
2036099336,6cyclictest0-21swapper/209:12:282
2036099331,8cyclictest4007-21ps08:00:182
2036099331,13cyclictest20616-21timerwakeupswit07:40:282
2036099331,11cyclictest30036-21cpu07:20:132
2036099331,11cyclictest20712-21cut10:45:232
2036099330,8cyclictest5176-21users10:00:282
2036099321,5cyclictest19897-21ssh11:55:442
20368993113,7cyclictest35966irq/58-eth008:45:133
2036099312,9cyclictest3743-21grep08:30:262
2036099312,10cyclictest1452-21apt-get11:30:032
2036099311,6cyclictest26369-21ntp_192.168.11408:20:262
2036099311,13cyclictest19411-21kthreadcore08:15:202
2036099311,12cyclictest3904-21apt-get12:25:022
2036099311,10cyclictest27640-21ssh10:15:142
2036099310,9cyclictest23704-21kthreadcore12:35:202
2036099310,11cyclictest22378-21ssh09:17:432
2034199311,22cyclictest32686-21sh10:00:100
2036099302,17cyclictest26621-21ssh10:50:132
2036099301,10cyclictest14815-21ls08:45:152
20341993011,14cyclictest12-21ksoftirqd/007:35:120
2036099292,24cyclictest0-21swapper/209:40:382
2036099292,24cyclictest0-21swapper/209:40:382
2036099292,20cyclictest0-21swapper/208:10:132
2036099291,6cyclictest32590-21ntp_states09:00:262
2035299295,17cyclictest753-21sendmail-mta07:45:141
2035299292,19cyclictest21938-21ssh10:29:441
2034199292,21cyclictest29518-21ssh09:56:440
20341992916,9cyclictest0-21swapper/010:40:140
20341992916,9cyclictest0-21swapper/010:40:140
2036899286,12cyclictest35966irq/58-eth010:30:133
2036899284,20cyclictest35966irq/58-eth010:40:143
2036899284,20cyclictest35966irq/58-eth010:40:143
2036899284,15cyclictest35966irq/58-eth009:15:123
2036099282,21cyclictest0-21swapper/211:47:442
2036099282,21cyclictest0-21swapper/208:40:142
2035299282,21cyclictest24530-21rm10:11:441
2035299282,20cyclictest4338-21ssh10:35:451
2034199282,21cyclictest30207-21ssh10:33:440
2034199282,20cyclictest16222-21ssh10:07:440
2034199282,18cyclictest844-21munin-node07:15:130
2035299273,21cyclictest4420-21ssh12:25:151
2035299272,19cyclictest30415-21ssh11:25:571
2034199272,16cyclictest10697-21kworker/u8:012:10:140
2034199271,19cyclictest28697-21ssh09:40:130
2034199271,19cyclictest28697-21ssh09:40:130
2036899265,11cyclictest0-21swapper/310:45:143
20368992621,2cyclictest0-21swapper/311:40:133
2036099264,12cyclictest27346irq/40-dwc2_hso07:55:132
2036099261,11cyclictest8305-21ssh10:40:022
2036099261,11cyclictest8305-21ssh10:40:022
2035299269,7cyclictest0-21swapper/108:10:121
2035299263,16cyclictest26605-21ssh10:50:121
2035299262,19cyclictest14233-21sh11:54:431
2035299261,19cyclictest24840-21sh09:20:141
2034199265,16cyclictest12-21ksoftirqd/012:05:110
2034199265,10cyclictest28726-21kthreadcore10:50:210
2034199263,19cyclictest483-21grep09:00:280
2034199262,19cyclictest12968-21ssh11:16:440
2034199262,18cyclictest13386-21ssh11:53:440
20368992515,7cyclictest35966irq/58-eth012:20:133
2036099252,9cyclictest0-21swapper/207:15:142
2035299256,9cyclictest21-21ksoftirqd/112:05:201
2035299253,13cyclictest20075-21ssh11:40:131
2035299252,17cyclictest6092-21diskmemload10:31:441
20352992514,5cyclictest0-21swapper/110:20:121
2035299251,19cyclictest3291-21ssh10:55:151
2035299251,16cyclictest5149-21ssh09:25:251
2034199252,18cyclictest14502-21ssh09:50:110
2036899246,11cyclictest35966irq/58-eth010:20:133
2036099242,16cyclictest0-21swapper/207:25:042
2035299243,4cyclictest21-21ksoftirqd/109:35:371
2034199247,12cyclictest12-21ksoftirqd/009:35:370
20341992416,4cyclictest16100-21ntp_offset11:00:250
20341992414,6cyclictest0-21swapper/007:25:130
2036899235,10cyclictest35966irq/58-eth012:25:153
2036899233,14cyclictest0-21swapper/309:50:123
2036899232,15cyclictest0-21swapper/310:10:123
2035299236,9cyclictest21-21ksoftirqd/111:47:141
2035299231,18cyclictest9580-21find12:10:121
2036899223,12cyclictest35966irq/58-eth010:35:143
2036899222,13cyclictest0-21swapper/312:35:143
2036099223,10cyclictest31377-21switchtime08:25:262
2035299223,12cyclictest21-21ksoftirqd/110:02:181
2035299222,15cyclictest22517-21ssh12:35:141
2035299221,7cyclictest11396-21kthreadcore08:05:221
20352992210,6cyclictest131rcu_preempt08:40:291
2034199223,8cyclictest12-21ksoftirqd/009:25:020
2034199223,15cyclictest0-21swapper/008:40:140
20341992214,5cyclictest27775-21mailstats12:00:280
2034199221,12cyclictest3781-21cut11:30:210
2036899213,12cyclictest0-21swapper/309:48:453
2036899213,12cyclictest0-21swapper/309:40:133
2036899213,12cyclictest0-21swapper/309:40:123
20368992113,5cyclictest0-21swapper/311:00:143
2036099211,16cyclictest4358-21kthreadcore09:05:222
2035299217,10cyclictest201rcuc/111:00:201
2035299215,3cyclictest131rcu_preempt12:17:351
2035299213,12cyclictest21-21ksoftirqd/108:55:141
2035299212,14cyclictest11173-21diskstats11:35:161
2035299212,11cyclictest14484-21ssh09:50:111
2035299211,2cyclictest131rcu_preempt11:14:081
2035299211,2cyclictest131rcu_preempt11:14:081
2035299211,14cyclictest28696-21sh09:40:131
2035299211,14cyclictest28696-21sh09:40:131
2035299211,14cyclictest20183-21ls07:40:221
20352992110,4cyclictest21-21ksoftirqd/110:47:041
2034199219,9cyclictest7971-21ntpq10:20:240
2034199217,11cyclictest8514-21proc_pri08:35:260
2034199212,15cyclictest3688-21apt-get08:00:120
20341992111,5cyclictest5296-21kthreadcore10:55:210
2036899208,8cyclictest35966irq/58-eth009:35:133
2036899208,6cyclictest35966irq/58-eth010:05:143
2036899203,12cyclictest0-21swapper/309:33:443
20368992013,4cyclictest0-21swapper/310:00:173
20368992012,5cyclictest35966irq/58-eth011:20:153
20368992011,6cyclictest0-21swapper/311:47:303
20368992011,5cyclictest0-21swapper/311:50:283
2036099205,12cyclictest14664-21kthreadcore07:35:212
2035299209,8cyclictest30117-21cut10:15:221
2035299209,8cyclictest21473-21apt-get11:05:141
2035299209,5cyclictest21-21ksoftirqd/111:24:501
2035299208,10cyclictest0-21swapper/112:32:151
2035299207,9cyclictest8256-21cron08:05:011
2035299206,6cyclictest131rcu_preempt09:17:401
2035299206,4cyclictest131rcu_preempt09:05:281
2035299204,10cyclictest21-21ksoftirqd/108:35:171
20352992013,5cyclictest0-21swapper/108:50:241
20352992013,3cyclictest0-21swapper/107:10:141
20352992012,6cyclictest201rcuc/107:25:021
2035299201,17cyclictest25528-21ps07:15:181
2034199206,11cyclictest17553-21tail07:40:180
20341992013,4cyclictest29849-21uname11:45:190
2034199201,16cyclictest21759-21kthreadcore07:10:210
20341992011,6cyclictest18420-21sshd08:15:180
2034199201,13cyclictest22700-21cut12:35:160
2036899193,11cyclictest0-21swapper/310:52:433
2036899192,10cyclictest0-21swapper/308:30:033
20368991912,4cyclictest0-21swapper/312:00:133
2036899191,13cyclictest0-21swapper/307:10:173
20368991910,6cyclictest0-21swapper/311:33:433
2036099192,9cyclictest24503-21fschecks_count08:55:162
2035299193,3cyclictest131rcu_preempt09:33:111
2035299192,9cyclictest0-21swapper/107:25:131
2035299192,5cyclictest21-21ksoftirqd/107:35:231
20352991914,3cyclictest0-21swapper/108:15:151
20352991913,4cyclictest0-21swapper/107:50:241
2034199195,6cyclictest0-21swapper/012:30:140
2034199191,9cyclictest8608-21fschecks_count08:05:150
2034199191,2cyclictest0-21swapper/009:49:030
2034199191,15cyclictest23149-21kthreadcore07:45:200
2034199191,15cyclictest14446-21sort09:30:280
2034199191,12cyclictest32374-21apt-key08:30:010
2036899189,6cyclictest35966irq/58-eth012:30:133
2036899183,11cyclictest0-21swapper/311:14:443
2036899183,11cyclictest0-21swapper/311:14:443
2036899183,10cyclictest0-21swapper/309:59:443
2036899182,10cyclictest0-21swapper/307:55:153
2036899182,10cyclictest0-21swapper/307:35:133
2036899182,10cyclictest0-21swapper/307:15:143
20368991811,4cyclictest35966irq/58-eth012:14:223
2036899181,12cyclictest0-21swapper/308:40:283
2036899181,11cyclictest0-21swapper/307:20:133
20368991810,6cyclictest0-21swapper/309:22:323
20368991810,5cyclictest35966irq/58-eth009:10:133
2036099181,9cyclictest20362-21latency_hist07:10:022
2035299188,8cyclictest29624-21cut12:20:191
2035299185,7cyclictest21-21ksoftirqd/107:55:191
2035299185,10cyclictest20378-21latency_hist07:10:021
20352991813,3cyclictest0-21swapper/108:25:151
20352991812,4cyclictest0-21swapper/108:30:121
20352991812,4cyclictest0-21swapper/108:20:191
20352991810,5cyclictest19225-21usage_Anybus11:55:321
2034199188,8cyclictest5095-21cat09:05:280
2034199187,8cyclictest22956-21apt-get08:20:150
2034199182,3cyclictest131rcu_preempt09:15:020
2034199181,8cyclictest16580-21tr08:45:210
20341991812,3cyclictest0-21swapper/007:50:160
2034199181,11cyclictest10906-21ls07:30:270
2036899177,7cyclictest0-21swapper/311:36:063
2036899172,9cyclictest0-21swapper/308:15:143
2036899172,10cyclictest0-21swapper/307:50:133
2036899172,10cyclictest0-21swapper/307:40:143
2036899172,10cyclictest0-21swapper/307:25:143
2036899171,12cyclictest0-21swapper/308:10:213
20352991712,3cyclictest0-21swapper/109:00:151
20352991711,4cyclictest0-21swapper/107:30:151
2035299171,13cyclictest16485-21kthreadcore08:45:211
2034199178,6cyclictest32608-21cut07:55:200
2034199172,4cyclictest0-21swapper/007:20:110
2034199171,13cyclictest32642-21apt-get08:30:130
2034199171,13cyclictest21666-21kthreadcore08:50:210
2036899162,9cyclictest0-21swapper/309:00:143
2036899161,9cyclictest0-21swapper/308:25:163
2034199164,10cyclictest13423-21fschecks_count08:10:150
2034199163,10cyclictest25022-21cut08:55:190
2036899157,5cyclictest35966irq/58-eth008:55:133
2036899151,11cyclictest0-21swapper/309:10:023
2036899151,10cyclictest0-21swapper/308:05:163
2034199151,10cyclictest20329-21cyclictest07:10:020
2036899132,8cyclictest35966irq/58-eth008:00:143
2036899122,7cyclictest0-21swapper/308:50:253
2036899122,7cyclictest0-21swapper/308:40:023
2036899122,7cyclictest0-21swapper/308:20:223
2036899122,7cyclictest0-21swapper/307:35:013
2036899111,7cyclictest0-21swapper/307:45:173
203689991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional