You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 08:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun Jan 11, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6383997045,16cyclictest28366irq/58-eth022:25:123
634899634,46cyclictest11068-21ssh23:45:130
634899545,39cyclictest5291-21ssh21:35:140
634899545,39cyclictest5291-21ssh21:35:130
6372995316,22cyclictest27-21ksoftirqd/219:10:122
636499515,39cyclictest1792-21ssh21:15:121
6372995021,23cyclictest27-21ksoftirqd/221:50:142
636499508,32cyclictest23163-21ssh00:10:121
6348995021,22cyclictest12-21ksoftirqd/020:30:120
636499464,31cyclictest17682-21ssh22:00:121
637299452,29cyclictest32037-21ssh00:15:112
636499451,40cyclictest26635-21apt-get22:05:131
634899453,32cyclictest5363-21ssh00:00:120
636499441,3cyclictest8670-21ssh21:55:131
634899444,26cyclictest6466-21ssh22:30:120
636499431,5cyclictest17527-21apt-get19:55:141
636499431,39cyclictest0-21swapper/122:50:131
638399423,20cyclictest28366irq/58-eth022:50:123
637299422,29cyclictest27-21ksoftirqd/219:40:122
636499421,35cyclictest5268-21apt-get21:35:131
636499421,35cyclictest5268-21apt-get21:35:121
636499411,37cyclictest0-21swapper/123:55:121
636499411,34cyclictest0-21swapper/121:45:121
634899414,4cyclictest131rcu_preempt21:25:130
637299403,30cyclictest22424-21ssh23:15:142
636499402,33cyclictest25774-21ssh23:35:111
636499402,33cyclictest20296-21apt-get21:05:141
636499401,35cyclictest0-21swapper/121:20:131
638399395,24cyclictest28366irq/58-eth022:45:143
638399395,24cyclictest28366irq/58-eth022:45:133
636499392,33cyclictest0-21swapper/123:00:121
636499391,4cyclictest0-21swapper/119:50:121
636499391,34cyclictest0-21swapper/123:20:131
638399383,21cyclictest28366irq/58-eth000:00:143
6383993820,13cyclictest28366irq/58-eth023:05:133
6372993823,9cyclictest27-21ksoftirqd/222:00:142
636499384,26cyclictest18786-21ssh22:55:121
636499381,31cyclictest18966-21apt-get20:30:121
636499381,2cyclictest28772-21apt-get21:30:131
636499380,2cyclictest0-21swapper/121:00:121
638399371,19cyclictest28366irq/58-eth019:55:133
637299374,22cyclictest18795-21ssh22:55:132
6372993720,11cyclictest0-21swapper/221:25:122
636499372,31cyclictest32027-21apt-get00:15:121
636499372,30cyclictest22353-21apt-get20:00:151
637299361,25cyclictest29931-21ssh22:25:132
636499361,30cyclictest30497-21tr20:40:201
636499361,30cyclictest0-21swapper/123:25:121
636499361,30cyclictest0-21swapper/123:25:111
636499360,4cyclictest11315-21apt-get19:15:101
6348993620,10cyclictest6684-21kworker/u8:100:20:140
6372993516,10cyclictest27-21ksoftirqd/220:25:122
636499351,30cyclictest6267-21tr19:40:221
636499351,30cyclictest26148-21/usr/sbin/munin00:30:181
636499351,30cyclictest12141-21kthreadcore20:55:191
636499351,30cyclictest11234-21kthreadcore00:20:211
636499351,29cyclictest32137-21apt-get21:50:131
636499350,31cyclictest12852-21wc20:20:261
634899357,18cyclictest12-21ksoftirqd/019:55:130
634899351,22cyclictest14262-21ssh21:40:120
638399342,18cyclictest28366irq/58-eth022:55:113
636499342,28cyclictest28463-21cron23:55:021
6364993413,18cyclictest14585-21if_err_eth121:40:161
636499341,30cyclictest7964-21ntp_states20:15:271
636499341,30cyclictest6382-21irqrtprio20:50:171
636499341,30cyclictest24416-21irqstats20:35:191
636499341,2cyclictest0-21swapper/100:05:141
636499341,2cyclictest0-21swapper/100:00:131
636499341,29cyclictest7154-21irqrtprio22:30:181
636499341,29cyclictest22152-21cat00:25:341
636499341,28cyclictest7298-21kthreadcore19:10:181
636499340,31cyclictest24368-21ntp_192.168.11419:25:231
636499340,30cyclictest4473-21ntp_states20:45:291
636499340,30cyclictest19917-21sort21:25:151
6348993416,8cyclictest12-21ksoftirqd/023:20:130
6383993321,7cyclictest28366irq/58-eth023:45:133
6372993311,12cyclictest27-21ksoftirqd/221:30:132
636499332,24cyclictest2707-21ssh00:35:151
636499332,24cyclictest14155-21apt-get20:25:131
636499331,3cyclictest0-21swapper/119:45:121
636499331,29cyclictest5805-21ntp_192.168.11423:40:261
636499331,29cyclictest21157-21cat23:30:311
636499331,29cyclictest14009-21cut22:15:211
636499331,28cyclictest30779-21sendmail_mailtr20:05:271
636499331,28cyclictest17003-21ntp_192.168.11423:10:261
636499331,27cyclictest26375-21cut19:30:191
636499330,2cyclictest0-21swapper/122:45:121
636499330,2cyclictest0-21swapper/122:45:111
636499330,2cyclictest0-21swapper/122:20:101
634899333,20cyclictest19896-21ssh23:50:120
634899333,19cyclictest2190-21diskmemload23:40:130
638399321,19cyclictest28366irq/58-eth022:10:133
638399321,19cyclictest28366irq/58-eth022:10:133
638399321,17cyclictest28366irq/58-eth021:55:133
637299322,24cyclictest26176-21ssh00:30:122
6372993211,16cyclictest27-21ksoftirqd/223:45:132
636499321,29cyclictest31564-21ssh22:43:171
636499321,28cyclictest606-21kthreadcore22:25:221
636499321,28cyclictest27109-21kthreadcore21:10:191
636499320,4cyclictest30601-21apt-get19:35:121
636499320,4cyclictest11060-21apt-get23:45:111
636499320,2cyclictest0-21swapper/122:35:191
636499320,2cyclictest0-21swapper/122:35:191
634899322,24cyclictest24417-21ssh22:40:130
6348993217,10cyclictest6334-21cyclictest23:30:120
6383993114,14cyclictest28366irq/58-eth000:25:133
636499311,9cyclictest16474-21ps19:20:161
636499311,27cyclictest3095-21ntp_192.168.11420:10:261
636499310,2cyclictest0-21swapper/122:10:121
636499310,2cyclictest0-21swapper/122:10:111
6348993121,6cyclictest12-21ksoftirqd/021:50:140
6383993017,10cyclictest0-21swapper/300:15:123
6383993016,8cyclictest0-21swapper/321:45:133
637299302,14cyclictest22346irq/40-dwc2_hso20:00:132
636499301,25cyclictest4223-21/usr/sbin/munin23:05:301
636499300,7cyclictest21948-21apt-get23:15:051
634899305,18cyclictest21016-21ssh22:20:110
634899301,21cyclictest14310-21ssh00:05:120
634899301,20cyclictest25769-21ssh23:35:110
634899301,20cyclictest23210-21ssh21:45:120
6383992921,3cyclictest28366irq/58-eth021:50:123
6383992920,3cyclictest0-21swapper/321:00:123
6383992918,7cyclictest0-21swapper/300:30:113
6383992916,9cyclictest0-21swapper/322:35:193
6383992916,9cyclictest0-21swapper/322:35:183
637299293,16cyclictest22346irq/40-dwc2_hso21:00:142
637299293,15cyclictest22346irq/40-dwc2_hso21:05:132
637299293,14cyclictest22346irq/40-dwc2_hso23:30:142
6383992823,2cyclictest0-21swapper/323:55:133
638399282,14cyclictest0-21swapper/321:15:123
6383992821,4cyclictest0-21swapper/320:25:133
6383992810,12cyclictest0-21swapper/322:40:123
637299282,21cyclictest10665-21ssh00:39:232
637299282,20cyclictest0-21swapper/220:15:152
637299282,10cyclictest8436-21ssh00:20:132
637299282,10cyclictest24872-21diskmemload23:40:132
6372992815,6cyclictest17322-21sh00:25:142
637299281,24cyclictest16445-21latency_hist23:30:012
637299281,24cyclictest16445-21latency_hist23:30:012
6348992820,3cyclictest0-21swapper/021:10:110
634899281,23cyclictest23145-21sh00:10:120
638399276,12cyclictest28366irq/58-eth022:15:143
637299273,20cyclictest22346irq/40-dwc2_hso23:50:122
637299272,19cyclictest0-21swapper/220:50:142
637299272,10cyclictest26646-21ssh22:05:112
637299271,15cyclictest25777-21diskmemload23:35:132
638399263,15cyclictest28366irq/58-eth021:20:133
638399262,14cyclictest28366irq/58-eth023:10:133
637299262,9cyclictest0-21swapper/220:20:142
637299262,7cyclictest27-21ksoftirqd/222:40:132
637299262,19cyclictest0-21swapper/219:25:142
637299262,10cyclictest0-21swapper/220:40:142
637299261,11cyclictest28854-21sh23:55:132
6372992611,12cyclictest0-21swapper/200:05:292
638399254,14cyclictest28366irq/58-eth000:20:123
6383992515,5cyclictest0-21swapper/300:05:143
6383992515,4cyclictest0-21swapper/323:20:133
6383992513,9cyclictest0-21swapper/323:35:123
637299252,19cyclictest0-21swapper/221:35:142
637299252,19cyclictest0-21swapper/221:35:142
6372992515,8cyclictest0-21swapper/220:05:242
637299251,11cyclictest13340-21ssh23:10:122
636499252,20cyclictest0-21swapper/119:10:011
6383992418,3cyclictest0-21swapper/320:35:123
6383992418,2cyclictest0-21swapper/323:25:123
6383992418,2cyclictest0-21swapper/323:25:113
6383992414,6cyclictest0-21swapper/323:40:143
637299243,12cyclictest22346irq/40-dwc2_hso19:35:112
6372992412,9cyclictest21063-21ssh21:42:512
6372992410,7cyclictest4257-21cron20:15:012
637299240,20cyclictest19018-21cpuspeed20:30:142
634899242,17cyclictest17314-21ssh00:25:130
6383992318,2cyclictest0-21swapper/319:25:123
6383992317,3cyclictest0-21swapper/319:20:103
6383992316,4cyclictest0-21swapper/323:00:133
6383992315,6cyclictest0-21swapper/321:35:133
6383992315,6cyclictest0-21swapper/321:35:123
6383992314,3cyclictest0-21swapper/322:20:093
6383992313,4cyclictest0-21swapper/323:50:143
637299238,12cyclictest14531-21grep20:55:282
6372992315,4cyclictest10833-21sh21:20:142
6372992310,6cyclictest0-21swapper/219:50:152
634899232,5cyclictest0-21swapper/000:33:120
6383992216,3cyclictest0-21swapper/320:45:123
6383992216,2cyclictest0-21swapper/319:35:113
6383992215,3cyclictest0-21swapper/319:40:133
6383992214,4cyclictest0-21swapper/319:50:123
6383992213,6cyclictest0-21swapper/300:10:113
6383992212,5cyclictest0-21swapper/319:45:113
637299229,5cyclictest1930-21cut23:20:232
637299225,5cyclictest924-21snmpd21:15:102
637299222,9cyclictest131rcu_preempt21:50:012
6372992211,9cyclictest0-21swapper/222:50:262
634899223,15cyclictest12145-21ssh22:15:150
634899222,16cyclictest3008-21apt-get19:40:110
6348992215,4cyclictest8681-21sh21:55:130
6348992214,4cyclictest12-21ksoftirqd/022:10:120
6348992214,4cyclictest12-21ksoftirqd/022:10:110
6348992211,8cyclictest7925-21df23:25:140
6348992211,8cyclictest7925-21df23:25:130
634899221,16cyclictest26658-21sh22:05:120
638399213,11cyclictest0-21swapper/321:30:123
6383992117,2cyclictest0-21swapper/321:40:123
6383992114,3cyclictest0-21swapper/320:10:153
6383992112,6cyclictest0-21swapper/323:30:123
6383992111,7cyclictest0-21swapper/322:30:193
6383992111,7cyclictest0-21swapper/320:40:153
637299218,5cyclictest24627-21kthreadcore00:10:202
637299214,14cyclictest27-21ksoftirqd/219:20:202
6372992112,7cyclictest0-21swapper/222:15:052
637299211,16cyclictest5042-21grep22:45:312
637299211,16cyclictest5042-21grep22:45:302
637299211,11cyclictest9509-21timerwakeupswit00:00:292
637299211,11cyclictest4819-21irqrtprio23:05:202
637299211,11cyclictest24912-21switchtime22:20:272
637299211,11cyclictest10130-21ntp_192.168.11422:30:242
637299211,10cyclictest7383-21unixbench_singl22:10:312
637299211,10cyclictest7383-21unixbench_singl22:10:302
637299211,10cyclictest27024-21latency20:35:212
637299210,9cyclictest27773-21cpuspeed23:00:142
634899217,4cyclictest131rcu_preempt20:25:290
634899212,5cyclictest0-21swapper/022:55:130
634899211,7cyclictest25105-21kthreadcore23:15:210
6348992114,4cyclictest12-21ksoftirqd/023:55:210
6348992111,7cyclictest11080-21cron19:15:010
634899211,16cyclictest28797-21sh21:30:130
634899211,13cyclictest3886-21ls23:05:020
638399203,10cyclictest0-21swapper/319:15:143
6383992010,7cyclictest0-21swapper/322:05:163
6383992010,7cyclictest0-21swapper/322:00:253
6383992010,7cyclictest0-21swapper/320:15:293
637299206,11cyclictest9908-21kthreadcore19:45:222
637299202,15cyclictest25905-21fschecks_count19:30:162
637299201,9cyclictest12813-21kthreadcore19:15:172
637299201,11cyclictest15471-21/usr/sbin/munin22:35:252
637299201,11cyclictest15471-21/usr/sbin/munin22:35:242
637299201,10cyclictest9330-21irqrtprio21:55:182
637299201,10cyclictest28812-21missed_timers21:10:232
637299200,15cyclictest2286-21kthreadcore20:45:222
634899208,10cyclictest12-21ksoftirqd/000:35:130
634899202,15cyclictest7857-21apt-get19:45:120
634899201,8cyclictest2019-21fschecks_time21:15:150
6348992013,4cyclictest4958-21irqstats23:05:190
634899201,2cyclictest0-21swapper/019:35:130
634899201,17cyclictest30396-21ls19:35:000
634899200,17cyclictest24584-21ls19:25:280
638399193,11cyclictest0-21swapper/300:36:243
638399192,10cyclictest28366irq/58-eth020:50:133
6383991915,2cyclictest0-21swapper/320:05:113
6383991911,6cyclictest0-21swapper/323:15:113
6383991911,5cyclictest0-21swapper/321:25:133
6383991911,4cyclictest0-21swapper/321:10:123
6383991911,2cyclictest0-21swapper/320:20:133
6383991910,7cyclictest0-21swapper/320:00:133
637299191,10cyclictest21040-21grep19:55:282
634899194,9cyclictest0-21swapper/022:50:110
634899192,13cyclictest15613-21df22:35:210
634899192,13cyclictest15613-21df22:35:200
634899192,13cyclictest0-21swapper/020:05:130
634899192,12cyclictest17679-21apt-get22:00:130
6348991912,4cyclictest11563-21kthreadcore21:20:200
6348991911,6cyclictest18956-21sshd23:11:230
634899191,15cyclictest8660-21kthreadcore20:50:210
634899191,14cyclictest31776-21apt-get20:10:030
638399188,7cyclictest0-21swapper/320:55:233
638399188,7cyclictest0-21swapper/320:30:173
638399188,7cyclictest0-21swapper/319:10:273
6383991814,2cyclictest0-21swapper/321:05:143
6383991813,2cyclictest0-21swapper/319:30:123
637299181,9cyclictest6397-21latency_hist19:10:002
634899187,8cyclictest23916-21cut20:00:210
634899184,11cyclictest19480-21missed_timers19:20:200
634899181,4cyclictest0-21swapper/022:45:450
634899181,4cyclictest0-21swapper/022:45:440
634899181,14cyclictest15482-21apt-get21:00:140
634899181,13cyclictest32047-21ssh00:15:120
634899181,13cyclictest30031-21df22:25:150
634899178,6cyclictest24878-21munin-run21:10:020
634899178,6cyclictest12-21ksoftirqd/020:40:160
634899175,9cyclictest28502-21latency_hist20:40:010
634899174,10cyclictest0-21swapper/020:55:110
634899172,12cyclictest12993-21kthreadcore19:15:180
634899170,9cyclictest4645-21fschecks_time20:15:160
634899163,10cyclictest9878-21irqrtprio20:20:190
634899162,9cyclictest0-21swapper/019:50:130
634899161,12cyclictest3114-21kthreadcore20:45:230
634899151,11cyclictest6403-21latency_hist19:10:000
638399137,3cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional