You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 19:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Sep 03, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2680199656,51cyclictest13107-21ssh09:10:130
26819996318,15cyclictest26759-21apt-get12:35:142
26819995212,30cyclictest0-21swapper/207:20:132
2681099482,4cyclictest131rcu_preempt09:30:161
2681999461,35cyclictest10456-21ssh11:50:152
2680199413,25cyclictest26650-21ssh11:05:120
2680199411,32cyclictest4744-21ssh10:35:130
2681999401,28cyclictest7833-21ssh09:25:132
2681999401,28cyclictest7833-21ssh09:25:122
2680199403,28cyclictest15389-21ssh10:05:130
2682899392,24cyclictest35966irq/58-eth010:45:153
2682899391,24cyclictest35966irq/58-eth009:05:143
26819993920,14cyclictest13091-21apt-get09:10:122
2681999391,22cyclictest10079-21ssh10:20:142
2682899374,21cyclictest35966irq/58-eth010:15:123
2681099371,26cyclictest26666-21ssh11:05:121
2681099371,12cyclictest30591-21grep12:35:301
2680199373,22cyclictest2229-21kworker/u8:110:20:130
2682899363,19cyclictest35966irq/58-eth009:35:113
2682899362,16cyclictest35966irq/58-eth011:25:113
26828993618,13cyclictest35966irq/58-eth010:00:113
2681099361,20cyclictest25897-21ssh09:35:111
2681099361,10cyclictest7524-21kthreadcore10:00:191
2680199364,23cyclictest24415-21ssh10:10:160
2681099351,9cyclictest31736-21kthreadcore08:55:181
2681099351,23cyclictest20716-21sh09:50:171
2681099351,10cyclictest24497-21kthreadcore11:20:221
2682899343,17cyclictest35966irq/58-eth012:25:123
2682899341,20cyclictest35966irq/58-eth012:20:123
2681099348,4cyclictest131rcu_preempt12:10:181
2681099347,4cyclictest12380-21kthreadcore08:00:211
2681099346,6cyclictest30346-21cut07:45:211
2681099343,6cyclictest0-21swapper/112:20:111
2681099342,10cyclictest4763-21ssh10:35:141
2681099341,9cyclictest3836-21irqrtprio11:10:181
2681099341,10cyclictest7220-21kthreadcore07:20:201
2681099341,10cyclictest30421-21ntp_states07:10:251
2681099341,10cyclictest10465-21irqcore10:20:191
2681099340,10cyclictest11613-21cut07:25:221
2681099340,10cyclictest11613-21cut07:25:211
2680199343,24cyclictest12-21ksoftirqd/011:25:120
26819993320,9cyclictest12230-21apt-get11:15:132
2681099338,4cyclictest964-21sh12:00:441
2681099338,4cyclictest964-21sh12:00:431
2681099337,21cyclictest21-21ksoftirqd/108:40:141
2681099333,27cyclictest28211-21ssh10:30:161
2681099332,9cyclictest12179-21ps10:55:251
2681099331,9cyclictest18088-21ssh12:30:221
2681099331,9cyclictest10937-21cut09:05:221
2681099331,29cyclictest31184-21ssh09:20:121
2681099331,10cyclictest6378-21cut12:05:201
2681099331,10cyclictest24130-21irqstats07:40:181
2681099331,10cyclictest21522-21kthreadcore07:35:221
2681099330,10cyclictest2851-21cut07:50:211
2682899322,19cyclictest35966irq/58-eth010:50:113
26819993224,3cyclictest11647-21apt-get09:45:122
26819993223,4cyclictest28201-21apt-get10:30:182
2681099327,4cyclictest25145-21cut10:45:211
2681099323,8cyclictest29926-21fschecks_time09:55:151
2681099321,9cyclictest7403-21kthreadcore08:30:171
2681099321,9cyclictest23423-21cut09:15:191
2681099321,9cyclictest19986-21grep11:35:301
2681099321,9cyclictest19687-21wc08:10:021
2681099321,9cyclictest1798-21switchtime11:25:271
2681099321,9cyclictest16483-21kthreadcore07:30:211
2681099321,28cyclictest11677-21ssh09:45:121
2681099321,10cyclictest3216-21cut08:25:201
2681099321,10cyclictest22977-21ssh12:15:121
2682899313,16cyclictest35966irq/58-eth011:30:173
2682899311,21cyclictest35966irq/58-eth012:10:173
2681999315,21cyclictest0-21swapper/212:29:342
2681999315,15cyclictest27346irq/40-dwc2_hso09:35:142
2681999312,24cyclictest5373-21ssh12:21:332
26819993113,10cyclictest0-21swapper/211:30:162
2681099311,9cyclictest3150-21pluginstate10:50:271
2681099311,12cyclictest1319-21if_eth110:15:171
2681099311,10cyclictest27262-21kthreadcore08:15:211
2681099311,10cyclictest23400-21proc_pri08:10:241
2680199312,17cyclictest12242-21ssh11:15:120
2682899303,18cyclictest35966irq/58-eth009:40:143
2682899303,18cyclictest35966irq/58-eth009:40:133
2681999303,16cyclictest27346irq/40-dwc2_hso08:10:112
26819993011,14cyclictest22962-21ssh12:15:122
2681099302,9cyclictest12628-21apt-get09:10:051
2681099302,8cyclictest844-21munin-node08:45:131
2681099302,10cyclictest18095-21tr10:05:211
2681099301,9cyclictest5547-21diskstats07:55:131
2681099301,9cyclictest5547-21diskstats07:55:121
2681099301,9cyclictest30820-21kthreadcore08:20:201
2681099301,9cyclictest19213-21cut11:00:231
2681099301,10cyclictest8549-21irqstats09:25:191
2681099301,10cyclictest8549-21irqstats09:25:181
26801993019,6cyclictest12618-21diskmemload11:55:110
2680199301,21cyclictest13850-21ssh10:40:130
2680199301,21cyclictest12618-21diskmemload09:55:140
2682899291,14cyclictest35966irq/58-eth010:05:133
2681999292,21cyclictest0-21swapper/209:00:132
26819992918,8cyclictest26965-21apt-get07:10:152
2681099294,6cyclictest5336-21kthreadcore09:00:211
2681099293,13cyclictest12618-21diskmemload10:40:251
2681099292,10cyclictest12315-21ssh11:15:131
2681099291,9cyclictest25372-21sh11:40:171
2681099291,9cyclictest15448-21munin-plugin-st11:35:011
2681099291,10cyclictest19332-21fschecks_count10:25:151
2681099291,10cyclictest19332-21fschecks_count10:25:141
2681099290,10cyclictest26461-21tr10:10:261
2680199292,21cyclictest21268-21ssh11:20:120
2680199291,24cyclictest0-21swapper/010:15:130
2681999283,14cyclictest27346irq/40-dwc2_hso07:45:122
2681999282,20cyclictest0-21swapper/208:45:052
26819992821,3cyclictest19339-21apt-get11:55:092
26819992815,8cyclictest15817-21apt-get11:35:122
2681099288,3cyclictest0-21swapper/111:49:501
2681099281,9cyclictest23312-21timerandwakeup11:55:271
2681099281,9cyclictest13623-21cut11:50:251
2682899272,13cyclictest35966irq/58-eth009:50:153
2682899270,18cyclictest35966irq/58-eth010:30:163
2681999272,19cyclictest22858-21ssh10:45:132
2681999272,16cyclictest27346irq/40-dwc2_hso10:35:132
2681999272,10cyclictest0-21swapper/207:35:142
26819992720,3cyclictest17696-21apt-get12:30:182
26819992719,4cyclictest31172-21apt-get09:20:132
26819992716,8cyclictest13970-21apt-get12:10:162
26819992712,9cyclictest27346irq/40-dwc2_hso08:55:132
26819992712,12cyclictest26064-21apt-get08:50:122
2681099271,9cyclictest5591-21kthreadcore09:40:231
2681099271,9cyclictest5591-21kthreadcore09:40:221
2680199273,17cyclictest12618-21diskmemload11:35:120
2680199271,18cyclictest4928-21ssh12:05:120
2682899261,17cyclictest0-21swapper/309:45:133
2681999265,13cyclictest0-21swapper/209:17:432
2681999264,6cyclictest190051ntp_kernel_err10:05:242
2681999264,10cyclictest0-21swapper/208:20:122
26819992616,8cyclictest28364-21apt-get12:00:142
26819992616,8cyclictest28364-21apt-get12:00:132
2680199262,18cyclictest854-21sshd12:22:340
2681999258,11cyclictest0-21swapper/211:20:112
2681999255,17cyclictest32608-21kthreadcore10:50:182
2681999255,12cyclictest27346irq/40-dwc2_hso08:30:122
2681999253,13cyclictest0-21swapper/209:51:282
2681999253,11cyclictest30748-21grep11:05:292
2681999252,4cyclictest5103-21ssh10:15:292
2681999252,19cyclictest0-21swapper/211:40:122
2681999252,19cyclictest0-21swapper/210:55:122
26819992513,9cyclictest0-21swapper/207:25:172
26819992513,9cyclictest0-21swapper/207:25:172
26819992512,10cyclictest27346irq/40-dwc2_hso09:30:152
26801992512,9cyclictest12-21ksoftirqd/009:25:130
26801992512,9cyclictest12-21ksoftirqd/009:25:120
2680199251,18cyclictest28205-21ssh10:30:150
2682899242,12cyclictest35966irq/58-eth011:15:123
2681999245,16cyclictest17116-21tr10:40:222
2681999244,14cyclictest27346irq/40-dwc2_hso08:05:122
2681999243,14cyclictest27346irq/40-dwc2_hso08:25:122
26819992417,3cyclictest3244-21apt-get11:10:132
26819992417,3cyclictest24412-21apt-get10:10:172
26819992412,8cyclictest11482-21apt-get08:35:142
2680199242,18cyclictest12618-21diskmemload10:25:120
2680199242,18cyclictest12618-21diskmemload10:25:110
2682899232,10cyclictest35966irq/58-eth007:35:123
26828992314,3cyclictest0-21swapper/307:40:123
2682899231,12cyclictest35966irq/58-eth011:05:113
2681999236,14cyclictest23013-21ssh11:01:082
2681999233,7cyclictest0-21swapper/211:27:022
2681999233,14cyclictest6351-21ssh10:00:112
26819992312,7cyclictest652-21apt-get07:50:132
2681999231,19cyclictest0-21swapper/210:25:112
2681999231,19cyclictest0-21swapper/210:25:102
2681099231,17cyclictest8633-21ssh12:25:121
2680199231,15cyclictest151rcuc/011:50:160
2682899229,9cyclictest35966irq/58-eth011:50:183
2681999229,5cyclictest7166-21kthreadcore12:05:222
2681999229,10cyclictest8128-21df09:05:162
2681999227,6cyclictest14168-21grep08:00:302
2681999223,8cyclictest27-21ksoftirqd/209:41:272
2681999223,8cyclictest27-21ksoftirqd/209:41:272
2681999222,13cyclictest27346irq/40-dwc2_hso08:15:132
2681999221,8cyclictest0-21swapper/209:57:332
2681999221,14cyclictest23479-21find07:40:122
2680199228,4cyclictest131rcu_preempt09:50:200
2680199223,6cyclictest5150-21ntp_states11:45:240
2680199222,15cyclictest28243-21ssh12:16:000
2680199222,14cyclictest13562-21ssh12:25:330
2680199221,17cyclictest12618-21diskmemload12:35:170
2680199221,16cyclictest26956-21find07:10:130
2682899212,13cyclictest35966irq/58-eth011:45:103
26828992111,6cyclictest0-21swapper/311:20:293
2681999212,13cyclictest27346irq/40-dwc2_hso07:55:112
2681999212,13cyclictest27346irq/40-dwc2_hso07:55:102
2681999211,4cyclictest0-21swapper/207:15:122
2681999211,13cyclictest2746-21kthreadcore11:45:182
26801992113,5cyclictest31863-21ssh10:50:110
2680199211,15cyclictest28394-21ssh12:00:150
2680199211,15cyclictest28394-21ssh12:00:140
2682899206,9cyclictest35966irq/58-eth010:40:113
2682899202,11cyclictest0-21swapper/309:15:043
26828992014,4cyclictest0-21swapper/310:10:313
26828992014,4cyclictest0-21swapper/309:30:333
26828992011,7cyclictest0-21swapper/311:55:103
26828992011,6cyclictest0-21swapper/311:40:123
26828992011,6cyclictest0-21swapper/310:20:333
26828992011,4cyclictest0-21swapper/312:39:403
2682899201,13cyclictest0-21swapper/312:05:133
2681999208,9cyclictest0-21swapper/207:30:162
26819992010,8cyclictest16298-21apt-get08:40:142
2680199209,7cyclictest0-21swapper/011:00:030
2680199206,11cyclictest16045-21cron08:40:010
2680199202,13cyclictest6273-21ntp_192.168.11409:40:260
2680199202,13cyclictest6273-21ntp_192.168.11409:40:250
26801992013,4cyclictest12618-21diskmemload11:35:020
2680199201,12cyclictest25749-21latency_hist08:50:010
2680199201,10cyclictest18438-21missed_timers08:05:220
2682899199,8cyclictest0-21swapper/311:11:193
2682899199,8cyclictest0-21swapper/309:29:173
2682899199,8cyclictest0-21swapper/309:29:163
2682899193,11cyclictest0-21swapper/312:31:003
26828991913,4cyclictest0-21swapper/312:15:173
26828991913,4cyclictest0-21swapper/310:55:243
26828991913,4cyclictest0-21swapper/309:55:293
26828991912,5cyclictest0-21swapper/312:02:053
26828991912,5cyclictest0-21swapper/312:02:043
26828991912,5cyclictest0-21swapper/309:23:363
26828991912,5cyclictest0-21swapper/307:20:143
26828991911,6cyclictest0-21swapper/308:00:273
26828991911,5cyclictest0-21swapper/311:35:143
2682899191,14cyclictest0-21swapper/308:50:153
26828991910,6cyclictest0-21swapper/309:12:043
2680199199,7cyclictest23617-21kthreadcore10:45:210
2680199198,3cyclictest131rcu_preempt12:30:220
2680199196,7cyclictest0-21swapper/007:30:140
2680199193,9cyclictest0-21swapper/011:10:130
2680199192,10cyclictest31193-21ssh09:20:130
2680199191,9cyclictest29349-21latency_hist08:20:010
26801991913,3cyclictest15355-21ntp_states09:45:260
2680199191,16cyclictest9143-21ls07:55:250
2680199191,16cyclictest9143-21ls07:55:250
2680199191,15cyclictest3166-21latency_hist09:00:010
2680199191,15cyclictest26918-21cut11:40:210
2680199191,15cyclictest19464-21kthreadcore08:40:220
2680199191,15cyclictest18083-21grep12:10:330
26801991911,5cyclictest17669-21kthreadcore09:30:240
2682899182,10cyclictest0-21swapper/309:00:143
2682899182,10cyclictest0-21swapper/308:55:133
2682899182,10cyclictest0-21swapper/308:45:133
2682899182,10cyclictest0-21swapper/307:30:143
26828991813,3cyclictest0-21swapper/310:35:253
26828991813,3cyclictest0-21swapper/310:28:143
26828991813,3cyclictest0-21swapper/310:28:133
2682899181,13cyclictest0-21swapper/308:40:293
2682899181,11cyclictest0-21swapper/308:10:013
2682899181,10cyclictest0-21swapper/308:20:243
2680199182,13cyclictest6471-21df_inode10:00:130
2680199182,12cyclictest29533-21ssh09:35:230
2680199182,11cyclictest22110-21ssh09:15:040
2680199182,11cyclictest12-21ksoftirqd/007:40:130
26801991812,4cyclictest0-21swapper/009:05:180
26801991812,4cyclictest0-21swapper/008:20:290
2680199181,15cyclictest7048-21tail09:00:270
26801991811,4cyclictest42-21kauditd07:25:010
2680199181,14cyclictest12210-21latency07:25:230
2680199181,14cyclictest12210-21latency07:25:220
2680199181,10cyclictest28414-21cut07:45:150
2682899172,9cyclictest0-21swapper/307:45:133
2682899172,10cyclictest0-21swapper/308:25:133
26828991712,3cyclictest0-21swapper/311:00:033
26828991711,4cyclictest0-21swapper/307:25:283
26828991711,4cyclictest0-21swapper/307:25:273
2681999171,13cyclictest26830-21latency_hist07:10:012
2681099171,11cyclictest14827-21ls08:35:221
2680199178,6cyclictest3511-21cut07:50:230
2680199174,3cyclictest131rcu_preempt07:20:010
2680199173,12cyclictest6832-21fschecks_count08:30:150
2680199172,10cyclictest0-21swapper/008:00:150
2680199171,9cyclictest18037-21sed11:00:190
26801991711,4cyclictest26793-21cyclictest07:10:010
2680199171,13cyclictest6649-21ls08:30:020
2680199170,9cyclictest29297-21latency08:50:240
26828991610,4cyclictest0-21swapper/308:10:283
2681099161,10cyclictest26199-21fschecks_count08:50:161
2680199161,12cyclictest22497-21kthreadcore08:10:200
2680199161,12cyclictest20887-21tr07:35:210
2682899151,8cyclictest0-21swapper/308:30:143
2681099151,10cyclictest31958-21fschecks_time07:15:161
2681099151,10cyclictest26920-21latency_hist07:10:031
2682899142,10cyclictest0-21swapper/308:15:133
2682899141,9cyclictest0-21swapper/307:55:123
2682899141,9cyclictest0-21swapper/307:55:113
2682899138,3cyclictest0-21swapper/307:50:163
2682899115,4cyclictest0-21swapper/307:10:273
2682899113,5cyclictest0-21swapper/307:15:133
2682899112,5cyclictest0-21swapper/308:35:133
2682899111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional