You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 10:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat Jul 18, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2341099775,43cyclictest20929-21ssh00:15:122
2339999552,46cyclictest13491-21ssh21:30:131
2339999522,41cyclictest20163-21ssh22:45:141
2339999512,5cyclictest131rcu_preempt20:30:111
2342299503,30cyclictest35966irq/58-eth022:00:133
2342299503,30cyclictest35966irq/58-eth022:00:133
2341099482,33cyclictest31724-21ssh21:40:132
2339099482,33cyclictest20143-21ssh22:45:120
2339999473,9cyclictest131rcu_preempt22:35:121
2339099461,39cyclictest11901-21ssh00:10:130
2339099461,36cyclictest22569-21rm21:35:130
2339099461,36cyclictest22569-21rm21:35:130
2341099442,24cyclictest17582-21ssh21:50:152
2339099443,35cyclictest29137-21ssh22:50:140
2339999431,35cyclictest21419-21ssh22:10:121
2339099434,31cyclictest9014-21diskmemload21:10:130
23422994223,11cyclictest35966irq/58-eth021:30:133
2339099422,30cyclictest17576-21ssh21:50:130
2342299414,22cyclictest35966irq/58-eth023:15:123
23410994120,13cyclictest0-21swapper/221:45:132
23422994017,11cyclictest0-21swapper/322:08:313
2341099401,17cyclictest9493-21sh21:10:132
2339099404,32cyclictest31715-21ssh21:40:110
23390994018,18cyclictest12-21ksoftirqd/022:00:140
23390994018,18cyclictest12-21ksoftirqd/022:00:130
2342299394,21cyclictest35966irq/58-eth022:30:123
2342299393,23cyclictest35966irq/58-eth020:40:133
23390993915,17cyclictest29978-21ssh00:20:130
23410993811,10cyclictest18181-21cat21:30:302
2339999381,31cyclictest26377-21ssh00:00:151
2339099381,29cyclictest22571-21ssh23:40:120
2342299371,21cyclictest35966irq/58-eth000:00:123
2342299371,20cyclictest35966irq/58-eth000:05:143
2339099362,24cyclictest25239-21ssh22:30:120
2341099352,27cyclictest29988-21ssh00:20:142
2339999356,21cyclictest9712-21ssh23:15:121
23399993528,4cyclictest22568-21sh21:35:131
23399993528,4cyclictest22568-21sh21:35:121
2339099352,24cyclictest30501-21ssh22:15:140
2342299342,17cyclictest35966irq/58-eth000:20:133
2342299341,22cyclictest35966irq/58-eth023:40:133
2341099332,29cyclictest17265-21ssh23:55:132
2341099332,29cyclictest17265-21ssh23:55:122
2341099332,24cyclictest28573-21ssh21:36:482
2341099332,24cyclictest28573-21ssh21:36:472
2342299325,13cyclictest35966irq/58-eth000:15:143
2339099323,19cyclictest9014-21diskmemload22:35:120
2339099321,20cyclictest6472-21ssh00:25:110
2342299313,17cyclictest35966irq/58-eth021:25:133
2341099312,18cyclictest27346irq/40-dwc2_hso23:05:122
23410993117,7cyclictest0-21swapper/219:50:122
2339099312,23cyclictest24030-21ssh23:05:140
2341099302,17cyclictest7195-21ssh22:20:132
2341099301,15cyclictest5851-21ssh22:55:142
2339999302,24cyclictest6499-21ssh00:25:131
23390993014,12cyclictest12-21ksoftirqd/023:50:150
2342299294,14cyclictest35966irq/58-eth022:55:143
2341099293,17cyclictest27346irq/40-dwc2_hso19:35:132
2341099293,16cyclictest27346irq/40-dwc2_hso00:35:122
2341099292,22cyclictest30500-21sh22:15:132
23399992915,8cyclictest9014-21diskmemload23:50:141
2339999291,22cyclictest4416-21ssh21:25:141
23390992918,8cyclictest12-21ksoftirqd/022:20:120
2341099285,14cyclictest27-21ksoftirqd/200:31:452
2341099284,14cyclictest27346irq/40-dwc2_hso00:10:142
2341099283,18cyclictest27346irq/40-dwc2_hso20:45:142
2341099283,18cyclictest27346irq/40-dwc2_hso20:45:132
2341099282,20cyclictest0-21swapper/219:15:152
2341099282,17cyclictest27346irq/40-dwc2_hso22:00:142
2341099282,17cyclictest27346irq/40-dwc2_hso22:00:132
2341099282,15cyclictest27346irq/40-dwc2_hso19:25:112
23410992817,5cyclictest261rcuc/222:38:472
2341099281,11cyclictest31596-21rm23:45:112
2339999283,14cyclictest4369-21ssh23:30:121
2339099285,20cyclictest12296-21ssh22:05:100
2342299273,16cyclictest0-21swapper/321:46:503
2341099278,4cyclictest27-21ksoftirqd/219:45:262
2341099277,9cyclictest27-21ksoftirqd/222:46:062
2341099274,8cyclictest27-21ksoftirqd/221:28:412
2341099272,21cyclictest31049-21ssh21:55:312
2341099272,15cyclictest27346irq/40-dwc2_hso19:20:122
2341099271,15cyclictest7797-21ssh00:05:332
2341099271,10cyclictest3035266ssh23:25:232
2339999273,19cyclictest3321-21ssh22:00:141
2339999273,19cyclictest3321-21ssh22:00:131
2339999272,3cyclictest131rcu_preempt20:35:131
2339999272,19cyclictest14193-21ssh21:46:311
2339999272,18cyclictest9014-21diskmemload21:15:131
23390992715,8cyclictest0-21swapper/000:35:120
2339099271,16cyclictest4394-21ssh23:30:130
23390992710,10cyclictest12-21ksoftirqd/021:45:140
2342299263,12cyclictest0-21swapper/321:15:123
2342299263,10cyclictest0-21swapper/300:25:123
2341099269,9cyclictest20918-21cron20:15:012
2341099266,8cyclictest27-21ksoftirqd/223:33:532
2341099264,9cyclictest27-21ksoftirqd/223:42:492
2341099262,18cyclictest21517-21ssh23:02:322
2339999262,18cyclictest9014-21diskmemload21:56:301
2339999262,17cyclictest11094-21ssh22:40:131
23390992613,9cyclictest1609-21sshd19:51:310
2342299257,12cyclictest35966irq/58-eth022:45:143
23422992516,6cyclictest35966irq/58-eth000:35:143
2341099257,8cyclictest27-21ksoftirqd/223:53:052
2341099257,11cyclictest27-21ksoftirqd/222:05:422
2341099256,9cyclictest27-21ksoftirqd/222:33:062
2341099256,8cyclictest27-21ksoftirqd/223:35:462
2341099256,13cyclictest27-21ksoftirqd/222:28:252
2341099255,11cyclictest131rcu_preempt21:15:222
2341099252,17cyclictest27346irq/40-dwc2_hso20:40:132
2341099252,11cyclictest27-21ksoftirqd/222:40:032
2341099251,12cyclictest27367-21munin-run23:25:012
2339999253,16cyclictest21-21ksoftirqd/100:05:121
2339999252,17cyclictest30502-21ssh22:15:141
2339099252,13cyclictest12-21ksoftirqd/020:15:130
2339099251,19cyclictest4527-21ssh21:25:160
2342299243,14cyclictest0-21swapper/321:36:483
2342299243,14cyclictest0-21swapper/321:36:473
23422992414,6cyclictest35966irq/58-eth021:55:143
2341099249,2cyclictest131rcu_preempt21:20:312
2341099248,13cyclictest10524-21timerwakeupswit20:00:282
2341099246,7cyclictest27-21ksoftirqd/219:40:162
2341099246,12cyclictest27-21ksoftirqd/200:03:222
2341099243,5cyclictest271ksoftirqd/221:05:192
2341099243,5cyclictest271ksoftirqd/221:05:192
2341099242,5cyclictest27-21ksoftirqd/220:55:202
23410992417,3cyclictest3133-21apt-get20:30:122
23410992414,4cyclictest261rcuc/222:10:572
2339999245,11cyclictest13473-21ssh23:35:121
2339999241,20cyclictest18722-21ssh23:20:131
2339999241,18cyclictest27767-21ssh23:25:141
23390992413,4cyclictest12-21ksoftirqd/021:57:570
2342299232,11cyclictest35966irq/58-eth022:20:113
2341099237,11cyclictest23501-21apt-get19:10:122
2341099236,8cyclictest8517-21kthreadcore00:25:192
2341099234,11cyclictest27346irq/40-dwc2_hso20:35:132
23410992314,7cyclictest261rcuc/223:15:532
2341099231,12cyclictest930-21snmpd20:20:132
2339999237,3cyclictest131rcu_preempt22:50:311
2339999236,3cyclictest131rcu_preempt21:20:201
2339999232,15cyclictest16230-21ssh22:25:131
2339099232,16cyclictest18707-21apt-get23:20:110
2339099232,15cyclictest19163-21ssh21:31:310
2339099231,17cyclictest27609-21ssh21:20:130
2342299228,8cyclictest35966irq/58-eth000:10:133
23422992210,8cyclictest0-21swapper/322:50:123
2341099228,6cyclictest131rcu_preempt23:10:522
2341099224,12cyclictest27-21ksoftirqd/222:53:292
2341099221,9cyclictest167-21systemd-network20:05:112
2341099221,13cyclictest27-21ksoftirqd/220:25:152
2341099221,11cyclictest24972-21tail20:15:292
2339999222,14cyclictest9314-21kworker/u8:322:08:311
2339999221,9cyclictest14617-21ls23:00:021
23399992214,5cyclictest0-21swapper/120:00:021
2339999221,18cyclictest23775-21ls23:05:021
2339099228,3cyclictest131rcu_preempt20:00:280
2339099228,11cyclictest0-21swapper/000:15:150
23390992213,5cyclictest31608-21ssh23:45:120
23390992212,6cyclictest22678-1kworker/u9:100:30:130
2339099221,17cyclictest739-21ls19:20:120
2339099221,16cyclictest647-21ssh23:10:130
23390992211,4cyclictest0-21swapper/022:55:280
2342299212,13cyclictest0-21swapper/321:51:303
2341099211,11cyclictest22544-21fschecks_time20:50:162
2339999219,9cyclictest211ksoftirqd/120:20:121
2339999212,14cyclictest24576-21sh00:35:141
23399992111,7cyclictest29240-21cron19:50:011
2339999211,17cyclictest17968-21tr21:50:191
2339999211,16cyclictest30771-21apt-get20:25:131
2339999211,15cyclictest24322-21ls00:35:011
2339999211,15cyclictest21-21ksoftirqd/120:05:131
2339999211,15cyclictest10395-21cut19:30:161
2339999211,15cyclictest10395-21cut19:30:161
2339999210,18cyclictest17716-21cut20:45:161
2339999210,18cyclictest17716-21cut20:45:151
23390992114,5cyclictest0-21swapper/000:00:160
2339099211,18cyclictest27196-21ls19:10:260
2339099211,17cyclictest26954-21apt-get20:55:000
2339099211,10cyclictest1694266ssh22:25:180
23422992011,7cyclictest0-21swapper/322:19:223
2342299201,12cyclictest0-21swapper/322:25:133
23422992010,8cyclictest0-21swapper/323:09:013
2341099208,9cyclictest0-21swapper/219:30:172
2341099208,9cyclictest0-21swapper/219:30:162
2341099208,4cyclictest0-21swapper/219:55:282
2341099203,4cyclictest27-21ksoftirqd/221:00:162
23399992011,5cyclictest9014-21diskmemload21:40:161
2339099209,3cyclictest131rcu_preempt00:05:280
2339099208,7cyclictest12-21ksoftirqd/019:30:140
2339099208,7cyclictest12-21ksoftirqd/019:30:130
2339099202,2cyclictest131rcu_preempt23:01:520
2339099201,2cyclictest131rcu_preempt23:30:000
2339099201,17cyclictest13415-21kthreadcore20:05:190
2339099201,15cyclictest17069-21ssh23:35:240
2339099201,15cyclictest11093-21sh22:40:140
2339099201,14cyclictest17270-21apt-get20:45:030
2339099201,10cyclictest15999-21kthreadcore19:35:180
2342299199,5cyclictest35966irq/58-eth023:50:153
2342299193,11cyclictest0-21swapper/323:03:313
2342299192,11cyclictest35966irq/58-eth021:40:113
23422991911,6cyclictest0-21swapper/323:35:213
2342299191,13cyclictest0-21swapper/323:30:123
2342299191,13cyclictest0-21swapper/320:15:233
2342299191,13cyclictest0-21swapper/319:30:023
2342299191,12cyclictest35966irq/58-eth022:40:113
2342299191,12cyclictest0-21swapper/319:55:303
23422991910,6cyclictest35966irq/58-eth023:25:153
23422991910,6cyclictest35966irq/58-eth021:20:133
2339999193,8cyclictest0-21swapper/123:55:131
2339999193,8cyclictest0-21swapper/123:55:121
2339999192,15cyclictest3541-21ssh23:10:241
2339999191,8cyclictest21646-21irqstats00:15:191
2339999191,4cyclictest0-21swapper/120:40:281
2339099196,3cyclictest12-21ksoftirqd/020:50:010
2339099196,3cyclictest12-21ksoftirqd/020:50:000
2339099192,2cyclictest131rcu_preempt22:10:210
2339099191,7cyclictest30992-21wc20:25:160
2339099191,3cyclictest131rcu_preempt23:15:110
23390991912,4cyclictest7913-21apt-get20:35:120
2342299189,6cyclictest0-21swapper/300:30:193
2342299188,6cyclictest0-21swapper/323:57:323
2342299188,6cyclictest0-21swapper/323:57:313
2342299187,6cyclictest0-21swapper/323:24:493
2342299182,10cyclictest0-21swapper/320:55:143
2342299182,10cyclictest0-21swapper/320:25:153
2342299182,10cyclictest0-21swapper/320:20:143
2342299182,10cyclictest0-21swapper/319:45:143
2342299182,10cyclictest0-21swapper/319:20:123
23422991812,4cyclictest0-21swapper/322:35:293
23422991811,4cyclictest0-21swapper/322:10:133
2342299181,12cyclictest0-21swapper/320:10:023
23422991810,6cyclictest0-21swapper/323:45:453
2339999187,2cyclictest0-21swapper/119:15:221
2339999184,3cyclictest131rcu_preempt22:30:121
2339999182,13cyclictest6420-21apt-get20:00:021
2339999181,4cyclictest131rcu_preempt23:05:161
23399991812,4cyclictest26294-21ntp_offset23:40:261
2339999181,14cyclictest30120-21kthreadcore20:55:221
2339999181,14cyclictest24976-21kthreadcore19:10:191
23399991811,4cyclictest16092-21uptime00:10:311
2339099188,8cyclictest25963-21apt-get20:20:130
2339099187,3cyclictest12-21ksoftirqd/021:10:020
2339099187,3cyclictest12-21ksoftirqd/021:10:010
2339099183,6cyclictest5487-21/usr/sbin/munin19:25:300
2339099181,3cyclictest131rcu_preempt23:58:500
2339099181,3cyclictest131rcu_preempt23:58:500
2339099181,3cyclictest131rcu_preempt19:15:160
23390991813,3cyclictest0-21swapper/021:00:190
2339099181,2cyclictest131rcu_preempt21:15:570
2339099181,15cyclictest29568-21kthreadcore20:55:230
23390991811,4cyclictest0-21swapper/020:30:210
2339099180,12cyclictest5531-21ls19:55:290
2342299179,6cyclictest0-21swapper/323:10:203
2342299172,12cyclictest0-21swapper/321:10:143
2342299172,10cyclictest0-21swapper/320:35:133
2342299172,10cyclictest0-21swapper/319:40:133
2342299172,10cyclictest0-21swapper/319:35:123
2342299171,11cyclictest0-21swapper/320:10:173
2341099171,8cyclictest23376-21cyclictest19:10:012
2339999179,6cyclictest3122-21ntp_states21:00:271
2339999175,9cyclictest24973-21kthreadcore20:50:231
2339999175,4cyclictest131rcu_preempt21:05:171
2339999175,4cyclictest131rcu_preempt21:05:161
2339999174,10cyclictest0-21swapper/119:50:191
2339999173,9cyclictest201rcuc/122:20:121
2339999171,5cyclictest30180-21fschecks_time00:20:151
2339999171,13cyclictest32702-21kthreadcore23:45:181
2339999171,10cyclictest23152-21latency19:40:211
2339999170,11cyclictest927-21ls19:20:161
2339099176,9cyclictest24692-21apt-get19:45:120
23390991712,3cyclictest0-21swapper/019:40:160
2342299162,9cyclictest0-21swapper/320:45:143
2342299162,9cyclictest0-21swapper/320:45:143
2339999168,5cyclictest21301-21fschecks_time20:15:171
2339999162,11cyclictest5853-21irqcore19:25:151
2339999162,11cyclictest17566-21kthreadcore19:35:201
2339999162,10cyclictest0-21swapper/120:10:121
2339999161,12cyclictest9728-21tune2fs21:10:171
2339099162,11cyclictest17860-21kthreadcore20:10:190
2342299152,9cyclictest0-21swapper/320:50:153
2342299151,7cyclictest0-21swapper/319:20:023
23422991510,3cyclictest0-21swapper/319:53:313
23422991510,3cyclictest0-21swapper/319:10:193
2342299142,9cyclictest0-21swapper/321:05:173
2342299142,9cyclictest0-21swapper/321:05:163
2342299142,7cyclictest0-21swapper/320:30:123
2342299141,11cyclictest0-21swapper/320:00:153
2339999141,10cyclictest23452-21latency_hist19:10:021
2339099141,10cyclictest23411-21latency_hist19:10:020
2342299131,9cyclictest0-21swapper/319:10:023
2342299131,8cyclictest0-21swapper/321:05:023
2342299123,7cyclictest0-21swapper/319:30:223
2342299123,7cyclictest0-21swapper/319:30:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional