You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-21 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed Oct 21, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1326996133,19cyclictest33366irq/58-eth011:25:133
131099563,6cyclictest131rcu_preempt09:25:131
131899462,36cyclictest20457-21ssh09:10:152
131899392,29cyclictest12912-21ssh10:15:272
131899392,19cyclictest30171-21grep11:00:282
130299373,29cyclictest1056-21rm11:40:130
1302993729,5cyclictest12-21ksoftirqd/011:35:140
130299370,3cyclictest131rcu_preempt10:50:120
1302993626,8cyclictest151rcuc/008:15:150
131099353,24cyclictest25953-21ssh11:00:121
130299343,25cyclictest14255-21ssh10:00:140
1302993416,14cyclictest11757-21sh11:10:150
130299341,30cyclictest23301-21ssh10:05:120
132699331,20cyclictest33366irq/58-eth007:35:133
130299336,23cyclictest28249-21ssh11:55:140
130299336,23cyclictest28249-21ssh11:55:130
130299333,4cyclictest131rcu_preempt07:45:140
1310993119,3cyclictest21-21ksoftirqd/107:20:111
130299313,23cyclictest3832-21ssh10:30:130
1302993123,6cyclictest12-21ksoftirqd/010:15:110
130299312,25cyclictest19395-21ssh09:45:110
130299311,26cyclictest15496-21sh11:30:140
131899301,22cyclictest19254-21ssh11:50:132
132699291,15cyclictest33366irq/58-eth012:30:153
132699291,15cyclictest33366irq/58-eth012:30:143
131899292,21cyclictest0-21swapper/208:25:162
131899291,25cyclictest28614-21kthreadcore07:35:232
131099292,6cyclictest131rcu_preempt12:00:121
131099292,21cyclictest7302-21sh10:13:271
131099292,21cyclictest22263-21grep10:20:271
130299292,23cyclictest29511-21ssh09:15:130
130299292,22cyclictest10145-21ssh11:45:130
1302992918,7cyclictest0-21swapper/008:00:140
132699283,18cyclictest0-21swapper/310:59:283
132699281,15cyclictest33366irq/58-eth011:10:133
131899282,21cyclictest25077-21ssh10:59:282
130299283,19cyclictest20750-21ssh11:15:150
131899273,18cyclictest0-21swapper/210:20:282
131899272,20cyclictest29383-21ssh10:43:292
131899272,19cyclictest22346irq/40-dwc2_hso07:20:122
131899272,10cyclictest383-21ssh10:26:282
131099278,4cyclictest131rcu_preempt11:00:001
130299271,3cyclictest131rcu_preempt10:20:130
1302992710,6cyclictest12-21ksoftirqd/007:10:120
132699265,15cyclictest0-21swapper/311:50:143
132699263,15cyclictest0-21swapper/311:00:273
1326992613,8cyclictest33366irq/58-eth009:10:143
131899263,18cyclictest0-21swapper/210:13:282
131899263,15cyclictest22346irq/40-dwc2_hso07:50:122
131899262,16cyclictest22748-21kthreadcore09:45:212
130299262,20cyclictest31046-21/usr/sbin/munin10:45:290
130299261,19cyclictest4855-21sh12:00:140
1302992611,11cyclictest664-21ntpd08:50:130
132699253,17cyclictest0-21swapper/310:53:273
131899256,3cyclictest131rcu_preempt08:15:162
131899253,14cyclictest6786-21ssh09:20:182
131899252,18cyclictest22089-21ssh12:09:272
131899252,14cyclictest28361-21timerandwakeup09:30:292
131899251,21cyclictest16162-21kthreadcore10:00:222
131899251,15cyclictest19064-21kthreadcore12:25:212
131899251,15cyclictest1063-21fschecks_count08:50:172
1318992510,11cyclictest28487-21kthreadcore12:30:232
1318992510,11cyclictest28487-21kthreadcore12:30:222
131099253,11cyclictest0-21swapper/109:10:141
130299255,13cyclictest20032-21diskmemload11:05:140
130299251,21cyclictest14601-21diskmemload11:29:280
131899243,18cyclictest5322-21kthreadcore08:20:172
131899242,19cyclictest21543-21kthreadcore08:35:252
1318992413,9cyclictest0-21swapper/211:05:292
1318992413,9cyclictest0-21swapper/207:15:152
131899241,20cyclictest1628-21unixbench_singl11:20:302
131899241,15cyclictest21846-21kthreadcore07:30:202
131899241,15cyclictest21846-21kthreadcore07:30:192
131099246,7cyclictest21-21ksoftirqd/111:35:131
130299242,19cyclictest1805-21ssh11:04:280
130299242,19cyclictest15123-21ssh12:22:280
130299241,18cyclictest29838-21apt-get11:20:120
132699234,13cyclictest0-21swapper/310:13:283
132699233,13cyclictest0-21swapper/310:20:283
132699233,13cyclictest0-21swapper/310:15:283
1326992311,9cyclictest0-21swapper/311:35:183
132699230,10cyclictest33366irq/58-eth007:25:123
131899237,13cyclictest4658-21kthreadcore07:45:242
131899232,6cyclictest0-21swapper/210:38:272
131899231,9cyclictest11733-21ssh11:10:142
131899231,13cyclictest12435-21kthreadcore09:00:212
1310992315,4cyclictest0-21swapper/107:50:141
130299235,10cyclictest32381-21ssh10:10:120
130299232,16cyclictest30152-21munin-run07:40:010
130299231,16cyclictest8820-21ssh09:55:260
130299231,16cyclictest27195-21ssh10:25:140
131899222,17cyclictest710-21cron08:00:012
131899222,12cyclictest17546-21ssh09:25:222
131899221,7cyclictest0-21swapper/209:55:152
1318992215,4cyclictest0-21swapper/209:55:012
131899221,19cyclictest13868-21needreboot09:40:242
131899221,18cyclictest3469-21kthreadcore07:10:202
131899221,13cyclictest9016-21tr10:50:212
131899221,13cyclictest12810-21grep12:20:302
131899221,12cyclictest7214-21sendmail_mailqu12:35:292
131899221,12cyclictest7214-21sendmail_mailqu12:35:292
1318992210,9cyclictest24829-21kthreadcore10:05:202
1318992210,4cyclictest12275-21kthreadcore11:45:202
131099223,17cyclictest4761-21kthreadcore12:35:221
131099223,17cyclictest4761-21kthreadcore12:35:221
1310992216,3cyclictest26328-21idleruntime-cro12:30:001
1310992212,7cyclictest28787-21hwlatdetect09:50:201
131099221,17cyclictest8602-21find12:20:131
1302992215,5cyclictest0-21swapper/009:50:220
1302992212,6cyclictest0-21swapper/008:05:280
132699219,9cyclictest0-21swapper/312:22:253
131899217,11cyclictest16287-21kthreadcore09:05:232
131899216,6cyclictest31317-1kworker/u9:012:15:182
131899214,9cyclictest26867-21latency_hist08:10:022
131899212,16cyclictest6364-21irqrtprio08:55:192
131899212,16cyclictest4244-21irqcore10:30:172
131899212,16cyclictest30531-21fschecks_count07:40:162
131899212,16cyclictest27003-21/usr/sbin/munin08:10:252
131899212,16cyclictest1877-21uname09:35:202
131899212,12cyclictest6669-21fschecks_time11:25:172
131899212,12cyclictest2459-21kthreadcore11:40:202
131899212,12cyclictest20510-21kthreadcore08:00:222
131899211,9cyclictest22960-21ssh12:10:142
131899211,16cyclictest30507-21kthreadcore09:15:202
131899211,13cyclictest26614-21ssh11:35:222
131899211,10cyclictest8832-21ssh12:00:302
131099218,3cyclictest131rcu_preempt12:10:021
131099217,3cyclictest131rcu_preempt11:40:211
131099213,16cyclictest22915-21rm10:05:001
131099213,15cyclictest20034-21awk09:10:001
131099212,17cyclictest9634-21sed08:55:301
131099212,16cyclictest3659-21kthreadcore07:45:211
131099212,16cyclictest20827-21fschecks_count07:30:161
131099212,16cyclictest20827-21fschecks_count07:30:151
131099212,16cyclictest19033-21fschecks_time08:35:191
131099212,16cyclictest10817-21fschecks_time09:00:171
1310992113,6cyclictest13681-21sshd08:28:281
130299219,8cyclictest22335-21if_err_eth010:40:170
130299212,14cyclictest16684-21ssh10:35:270
1302992115,3cyclictest0-21swapper/012:35:140
1302992115,3cyclictest0-21swapper/012:35:130
1302992112,3cyclictest20032-21diskmemload09:30:130
132699209,6cyclictest0-21swapper/309:45:303
132699202,10cyclictest0-21swapper/307:50:133
1326992011,7cyclictest0-21swapper/311:20:223
131899203,15cyclictest29516-21kthreadcore08:45:182
131899202,16cyclictest21121-21ssh11:15:182
131899202,10cyclictest28403-21fschecks_count11:55:162
131899202,10cyclictest28403-21fschecks_count11:55:152
1318992014,4cyclictest964-21lldpd08:30:182
131899201,11cyclictest16729-21kthreadcore07:25:182
131899201,10cyclictest24013-21sed08:40:192
131899201,10cyclictest16533-21kthreadcore11:30:212
1318992010,8cyclictest7405-21sh10:50:012
131099209,3cyclictest21-21ksoftirqd/111:20:271
131099204,9cyclictest22941-21ssh12:10:121
131099202,16cyclictest32750-21usage_Anybus11:55:311
131099202,16cyclictest32750-21usage_Anybus11:55:301
131099202,16cyclictest31732-21latency_hist08:15:021
131099202,16cyclictest2179-21kthreadcore08:15:231
131099202,16cyclictest21375-21kthreadcore11:50:221
131099202,15cyclictest9171-21kthreadcore09:20:221
131099202,15cyclictest27264-21kthreadcore09:30:231
131099202,15cyclictest26583-21kthreadcore10:05:221
131099202,15cyclictest16826-21kthreadcore08:30:211
131099201,6cyclictest30508-21ntp_states12:30:291
131099201,6cyclictest30508-21ntp_states12:30:281
131099201,6cyclictest20415-21sed11:15:021
1310992013,4cyclictest0-21swapper/111:10:011
131099201,16cyclictest9924-21timerandwakeup07:15:271
130299206,4cyclictest32343-21irqcore12:15:160
130299204,7cyclictest0-21swapper/010:55:140
130299204,12cyclictest0-21swapper/007:55:120
130299203,12cyclictest0-21swapper/008:40:150
130299202,11cyclictest10255-21ssh09:40:140
130299201,14cyclictest1142-21sh09:35:150
130299200,17cyclictest14688-21sed12:05:200
132699199,8cyclictest0-21swapper/312:27:173
132699198,9cyclictest0-21swapper/310:35:143
132699198,6cyclictest0-21swapper/311:30:453
132699193,11cyclictest0-21swapper/312:09:283
1326991912,5cyclictest33366irq/58-eth011:15:153
1326991911,5cyclictest33366irq/58-eth010:30:123
132699191,14cyclictest0-21swapper/308:45:153
132699191,12cyclictest0-21swapper/310:40:163
132699191,12cyclictest0-21swapper/309:40:173
1326991910,6cyclictest33366irq/58-eth012:10:133
1326991910,6cyclictest33366irq/58-eth009:30:143
131099199,7cyclictest16630-21ntpq10:35:271
131099193,14cyclictest9591-21tr11:25:221
131099193,13cyclictest2680-21kthreadcore09:35:201
131099192,15cyclictest21959-21kthreadcore11:15:201
131099192,15cyclictest12607-21fschecks_time07:55:171
131099192,14cyclictest6284-21kthreadcore08:20:181
131099192,14cyclictest28546-21kthreadcore07:35:221
131099192,14cyclictest22075-21kthreadcore09:45:191
131099192,14cyclictest15874-21acpi07:25:121
131099192,12cyclictest16546-21kworker/u8:210:40:141
1310991911,6cyclictest4420-21sshd10:46:291
131099191,14cyclictest19205-21ssh11:30:281
1310991910,7cyclictest18929-21cron09:45:011
130299194,12cyclictest23208-21ssh12:10:170
130299191,15cyclictest21295-21irqrtprio07:30:190
130299191,15cyclictest21295-21irqrtprio07:30:180
130299191,15cyclictest17633-21sshd08:30:280
130299191,14cyclictest6196-21apt-get09:20:110
132699189,7cyclictest0-21swapper/312:35:333
132699189,7cyclictest0-21swapper/312:35:333
132699189,7cyclictest0-21swapper/311:59:503
132699189,7cyclictest0-21swapper/311:59:503
132699189,6cyclictest0-21swapper/310:28:263
132699189,6cyclictest0-21swapper/310:05:533
132699189,6cyclictest0-21swapper/309:38:183
132699188,7cyclictest0-21swapper/311:44:113
132699182,11cyclictest33366irq/58-eth009:55:143
132699182,10cyclictest0-21swapper/308:10:153
1326991813,2cyclictest0-21swapper/308:50:133
1326991810,6cyclictest0-21swapper/309:23:033
131899182,8cyclictest1370-21latency_hist07:10:022
131899182,8cyclictest1370-21latency_hist07:10:012
131099189,7cyclictest28531-21apt-get08:45:131
131099189,6cyclictest29579-21kthreadcore10:25:221
131099189,6cyclictest21-21ksoftirqd/110:15:121
131099183,12cyclictest6650-21cut09:55:221
131099182,13cyclictest18378-21kthreadcore08:00:211
131099181,14cyclictest4220-21irqcore10:30:161
131099181,12cyclictest10246-21ssh11:45:131
1310991810,6cyclictest10932-21kthreadcore10:50:231
1302991812,4cyclictest0-21swapper/008:20:210
1302991811,5cyclictest23243-21ssh09:29:110
1302991811,5cyclictest0-21swapper/009:05:210
130299181,13cyclictest630-21Async17:24:400
130299181,12cyclictest20807-21if_eth109:10:180
130299180,3cyclictest131rcu_preempt12:27:410
132699179,6cyclictest0-21swapper/312:00:063
132699179,6cyclictest0-21swapper/311:48:393
132699179,5cyclictest33366irq/58-eth011:05:143
132699179,5cyclictest33366irq/58-eth010:45:143
132699178,7cyclictest0-21swapper/310:00:103
132699177,7cyclictest0-21swapper/309:25:293
132699176,5cyclictest0-21swapper/312:15:293
132699172,9cyclictest0-21swapper/307:40:153
132699172,10cyclictest0-21swapper/308:35:043
132699172,10cyclictest0-21swapper/308:25:153
132699171,13cyclictest0-21swapper/308:55:173
1326991710,4cyclictest0-21swapper/309:50:163
131099173,12cyclictest4272-21missed_timers08:50:231
131099172,13cyclictest24273-21kthreadcore08:05:211
131099172,12cyclictest25888-21kthreadcore08:40:211
131099171,9cyclictest3176-21ntp_192.168.11412:15:221
131099171,13cyclictest742-21kthreadcore07:40:221
131099171,12cyclictest29585-21sort09:15:141
130299178,6cyclictest13896-21latency09:00:220
130299175,4cyclictest12-21ksoftirqd/012:30:200
130299175,4cyclictest12-21ksoftirqd/012:30:190
130299173,12cyclictest31240-21/usr/sbin/munin08:10:310
130299172,9cyclictest12072-21kthreadcore08:25:220
130299171,7cyclictest30605-21hwlatdetect07:40:170
130299171,12cyclictest19245-21apt-get11:50:120
1326991610,4cyclictest0-21swapper/309:15:303
131099161,12cyclictest1575-21tune2fs07:10:141
130299164,9cyclictest20152-21kthreadcore08:35:220
130299162,11cyclictest16774-21kthreadcore07:25:180
130299162,10cyclictest0-21swapper/008:45:130
130299162,10cyclictest0-21swapper/007:20:130
130299160,13cyclictest6271-21irqrtprio08:55:200
132699151,8cyclictest0-21swapper/308:00:173
131099151,11cyclictest1395-21latency_hist07:10:021
130299151,11cyclictest7601-21apt-get07:50:140
130299151,11cyclictest1335-21cat07:10:020
130299151,11cyclictest1335-21cat07:10:010
132699149,3cyclictest0-21swapper/308:15:193
132699149,3cyclictest0-21swapper/308:07:283
132699149,3cyclictest0-21swapper/307:19:213
131099141,10cyclictest1372-21latency_hist07:10:021
132699137,3cyclictest0-21swapper/309:05:093
132699131,10cyclictest0-21swapper/307:20:213
132699127,3cyclictest0-21swapper/307:30:233
132699127,3cyclictest0-21swapper/307:30:233
132699123,7cyclictest0-21swapper/309:00:273
132699123,6cyclictest0-21swapper/307:45:173
132699121,8cyclictest0-21swapper/308:45:023
132699111,7cyclictest0-21swapper/308:30:163
132699111,7cyclictest0-21swapper/308:20:103
132699111,7cyclictest0-21swapper/307:55:163
132699111,4cyclictest0-21swapper/307:10:133
13269991,6cyclictest0-21swapper/307:10:023
13269991,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional