You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-11 - 11:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Sep 11, 2026 00:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
121399643,15cyclictest27-21ksoftirqd/223:05:142
118999644,51cyclictest5474-21ssh00:20:120
121399576,27cyclictest9061-21ssh23:10:122
121399491,32cyclictest5420-21ssh21:20:142
122599483,27cyclictest35966irq/58-eth020:50:143
122599483,27cyclictest35966irq/58-eth020:50:143
121399453,26cyclictest26989-21ssh23:20:132
120499451,34cyclictest30544-21ssh23:40:121
122599446,23cyclictest35966irq/58-eth022:00:133
118999441,33cyclictest5513-21ssh22:50:170
1213994120,14cyclictest0-21swapper/200:15:122
122599402,25cyclictest35966irq/58-eth022:45:133
120499401,30cyclictest19884-21ssh22:40:131
122599393,23cyclictest35966irq/58-eth022:15:123
122599393,18cyclictest35966irq/58-eth021:05:143
121399392,29cyclictest3550-21ntp_192.168.11421:35:252
120499381,33cyclictest16289-21ssh22:20:121
120499381,30cyclictest21564-21ssh23:35:131
1204993717,9cyclictest21-21ksoftirqd/119:40:131
122599363,19cyclictest35966irq/58-eth022:10:123
122599361,22cyclictest35966irq/58-eth023:45:123
1225993612,14cyclictest35966irq/58-eth023:35:133
122599361,20cyclictest35966irq/58-eth021:45:123
121399361,17cyclictest0-21swapper/220:55:132
120499366,8cyclictest131rcu_preempt23:20:131
121399359,6cyclictest27-21ksoftirqd/222:25:152
121399354,23cyclictest14419-21sh22:55:142
120499352,26cyclictest26994-21ssh21:50:121
120499352,22cyclictest8991-21ssh21:40:141
122599343,20cyclictest35966irq/58-eth023:10:123
1213993418,11cyclictest0-21swapper/222:20:142
1189993428,3cyclictest12-21ksoftirqd/022:45:130
120499331,25cyclictest854-21sshd00:30:251
120499331,25cyclictest854-21sshd00:30:251
120499330,26cyclictest25328-21ssh22:25:141
118999333,15cyclictest7249-21ssh22:15:110
1189993324,3cyclictest12-21ksoftirqd/021:00:130
122599324,17cyclictest35966irq/58-eth020:55:133
122599322,22cyclictest35966irq/58-eth019:30:123
121399324,17cyclictest27346irq/40-dwc2_hso21:10:112
1213993214,10cyclictest27-21ksoftirqd/221:40:142
120499321,28cyclictest23383-21ssh23:00:131
122599313,12cyclictest35966irq/58-eth023:40:113
122599312,19cyclictest35966irq/58-eth022:40:133
1225993110,14cyclictest35966irq/58-eth000:05:123
121399313,19cyclictest27346irq/40-dwc2_hso19:45:122
120499313,26cyclictest25269-21ssh23:55:161
121399302,15cyclictest23407-21ssh23:00:142
1213993013,7cyclictest261rcuc/200:00:132
121399301,19cyclictest16260-21sh23:50:152
120499303,22cyclictest12631-21ssh22:00:131
120499301,26cyclictest32522-21ssh00:35:141
120499301,26cyclictest32522-21ssh00:35:141
122599291,5cyclictest0-21swapper/323:05:143
1225992913,8cyclictest0-21swapper/321:15:143
121399293,20cyclictest0-21swapper/219:35:142
121399291,16cyclictest0-21swapper/223:30:152
118999292,17cyclictest0-21swapper/022:25:150
1189992913,5cyclictest131rcu_preempt20:35:130
118999290,2cyclictest131rcu_preempt22:40:140
122599287,3cyclictest0-21swapper/323:24:083
1225992811,10cyclictest35966irq/58-eth000:15:133
121399283,17cyclictest27346irq/40-dwc2_hso20:20:122
120499282,19cyclictest14298-21ssh21:25:141
120499282,19cyclictest14298-21ssh21:25:141
118999283,22cyclictest21674-21ssh22:05:120
122599273,20cyclictest35966irq/58-eth000:20:133
122599272,6cyclictest0-21swapper/300:31:263
122599272,6cyclictest0-21swapper/300:31:263
122599272,10cyclictest0-21swapper/322:05:303
122599271,14cyclictest35966irq/58-eth000:00:133
121399275,13cyclictest27346irq/40-dwc2_hso20:00:132
121399274,13cyclictest27346irq/40-dwc2_hso20:50:132
121399274,13cyclictest27346irq/40-dwc2_hso20:50:132
121399272,16cyclictest27346irq/40-dwc2_hso20:10:132
121399272,11cyclictest0-21swapper/200:30:142
121399272,11cyclictest0-21swapper/200:30:142
118999272,19cyclictest491-21ssh21:51:540
118999272,19cyclictest18723-21ssh22:01:580
118999272,19cyclictest17990-21ssh23:15:110
1189992711,10cyclictest3720-21ntp_192.168.11400:35:250
1189992711,10cyclictest3720-21ntp_192.168.11400:35:240
122599267,6cyclictest35966irq/58-eth023:30:163
122599267,6cyclictest35966irq/58-eth023:00:143
122599267,6cyclictest0-21swapper/323:16:103
121399266,15cyclictest0-21swapper/221:28:252
121399266,15cyclictest0-21swapper/221:28:252
121399264,14cyclictest27346irq/40-dwc2_hso20:05:132
121399263,17cyclictest27346irq/40-dwc2_hso21:05:132
121399263,17cyclictest27346irq/40-dwc2_hso19:25:112
118999265,18cyclictest9077-21ssh23:10:130
118999264,20cyclictest23295-21ssh21:30:130
118999264,15cyclictest28832-21ssh21:15:150
118999262,18cyclictest27214-21/usr/sbin/munin00:30:250
118999262,18cyclictest27214-21/usr/sbin/munin00:30:250
122599259,3cyclictest0-21swapper/321:53:523
122599258,3cyclictest0-21swapper/321:33:503
122599251,8cyclictest0-21swapper/321:40:023
1225992510,4cyclictest0-21swapper/300:25:343
121399255,7cyclictest27-21ksoftirqd/222:33:312
121399255,7cyclictest27-21ksoftirqd/222:33:312
121399253,3cyclictest131rcu_preempt19:15:152
1213992514,9cyclictest0-21swapper/222:05:172
121399251,17cyclictest4350-21kthreadcore21:55:182
120499258,11cyclictest1177-21cyclictest23:30:151
1204992517,5cyclictest0-21swapper/122:10:131
118999252,19cyclictest10905-21ssh22:35:140
122599249,4cyclictest0-21swapper/323:57:303
122599249,12cyclictest0-21swapper/321:26:543
122599249,12cyclictest0-21swapper/321:26:533
122599241,6cyclictest0-21swapper/320:40:213
121399242,16cyclictest0-21swapper/200:25:252
121399242,14cyclictest0-21swapper/219:40:122
121399241,13cyclictest23772-21switchtime00:10:312
120499242,16cyclictest3618-21ssh21:55:131
120499241,19cyclictest7192-21ssh23:45:131
122599239,12cyclictest0-21swapper/300:12:513
122599238,6cyclictest0-21swapper/322:37:173
122599234,5cyclictest35966irq/58-eth022:55:223
122599233,17cyclictest0-21swapper/319:20:133
122599231,8cyclictest0-21swapper/322:20:183
122599231,7cyclictest0-21swapper/320:05:013
121399239,11cyclictest4791-21cut23:25:192
121399236,13cyclictest27-21ksoftirqd/221:15:132
121399233,3cyclictest131rcu_preempt23:45:242
121399233,15cyclictest27-21ksoftirqd/223:35:142
121399233,11cyclictest16806-21head22:00:312
121399233,11cyclictest13492-21kthreadcore19:55:182
121399231,7cyclictest19401-21diskmemload21:33:252
1213992311,9cyclictest0-21swapper/223:15:302
121399231,12cyclictest9417-21grep00:20:292
120499234,13cyclictest0-21swapper/122:50:161
120499232,13cyclictest14502-21ssh00:25:141
1204992315,5cyclictest0-21swapper/123:10:021
120499231,19cyclictest25427-21find19:35:121
120499231,16cyclictest28794-21ssh00:15:131
118999239,4cyclictest131rcu_preempt23:05:210
118999232,17cyclictest3606-21ssh23:25:120
118999231,19cyclictest14508-21ssh00:25:140
122599227,4cyclictest0-21swapper/322:29:493
122599224,11cyclictest35966irq/58-eth020:20:123
122599221,7cyclictest0-21swapper/321:20:243
1225992212,7cyclictest35966irq/58-eth021:55:133
1225992210,9cyclictest0-21swapper/300:35:193
1225992210,9cyclictest0-21swapper/300:35:193
121399228,11cyclictest29094-21grep22:45:172
121399222,9cyclictest32538-21ssh00:35:142
121399222,9cyclictest32538-21ssh00:35:142
121399222,15cyclictest0-21swapper/219:11:262
121399221,9cyclictest1808666ssh21:45:142
121399221,12cyclictest27446-21cut23:55:232
121399221,12cyclictest14921-21timerwakeupswit00:05:302
121399221,11cyclictest20553-21sed22:40:192
121399220,6cyclictest10238-21ls21:00:162
120499221,16cyclictest10421-21ssh00:20:341
1204992210,9cyclictest26585-21cron23:20:011
118999229,11cyclictest12-21ksoftirqd/019:30:120
1189992213,6cyclictest30019-21cron19:40:010
1189992210,8cyclictest1177-21cyclictest00:19:250
122599218,9cyclictest35966irq/58-eth019:45:103
122599218,3cyclictest0-21swapper/322:50:033
122599212,6cyclictest0-21swapper/321:41:253
121399218,4cyclictest19127-21kthreadcore20:35:182
121399217,11cyclictest20717-21df_inode19:30:142
121399216,8cyclictest27937-21sh21:50:202
121399212,9cyclictest8111-21kthreadcore22:50:262
121399211,9cyclictest32694-21cut23:40:202
121399211,6cyclictest0-21swapper/219:50:132
1213992114,5cyclictest0-21swapper/219:20:182
1213992111,8cyclictest0-21swapper/220:45:182
121399211,16cyclictest13754-21fschecks_time20:30:162
120499212,15cyclictest18390-21apt-get20:35:121
120499211,16cyclictest10008-21apt-get21:00:111
120499211,15cyclictest7226-21find22:15:101
118999213,15cyclictest5202-21kthreadcore20:20:190
118999212,15cyclictest1829-21rm23:40:250
1189992112,7cyclictest5596-21kthreadcore21:55:210
118999211,16cyclictest12-21ksoftirqd/020:40:120
1189992110,8cyclictest9235-21/usr/sbin/munin21:40:180
1225992011,6cyclictest0-21swapper/321:10:143
121399209,8cyclictest27-21ksoftirqd/222:10:122
121399208,4cyclictest27-21ksoftirqd/220:25:262
121399206,11cyclictest31707-21fschecks_time20:15:172
121399202,15cyclictest23701-21irqrtprio20:40:182
121399201,8cyclictest10913-21sh22:35:152
121399201,7cyclictest9803-21cut22:15:202
120499207,6cyclictest27065-21tr20:10:161
120499205,11cyclictest21-21ksoftirqd/100:02:171
1204992013,5cyclictest0-21swapper/119:45:261
1204992013,5cyclictest0-21swapper/119:10:231
1204992012,5cyclictest19398-21latency_hist21:10:021
120499201,10cyclictest9512-21/usr/sbin/munin21:20:311
1204992010,4cyclictest131rcu_preempt22:05:201
118999209,6cyclictest0-21swapper/000:01:250
118999201,3cyclictest0-21swapper/021:37:420
1189992013,6cyclictest73-21kswapd022:10:120
1189992013,4cyclictest21736-21fschecks_count23:35:150
1189992012,6cyclictest0-21swapper/023:30:310
1189992011,7cyclictest19778-21ssh21:10:110
1189992010,7cyclictest14394-21df_inode21:25:150
1189992010,7cyclictest14394-21df_inode21:25:150
122599193,3cyclictest0-21swapper/319:40:223
122599192,11cyclictest0-21swapper/320:35:143
122599191,12cyclictest0-21swapper/319:15:153
121399191,9cyclictest1346-21latency_hist19:10:012
120499193,5cyclictest131rcu_preempt21:15:131
120499192,11cyclictest3558-21ssh21:35:251
120499191,9cyclictest15479-21kthreadcore20:30:201
120499191,15cyclictest14522-21ssh00:05:261
120499191,14cyclictest23504-21rm00:10:281
120499191,12cyclictest16257-21ssh23:50:151
120499191,10cyclictest24253-21kthreadcore21:30:201
120499190,5cyclictest53646usb-storage23:10:111
118999199,7cyclictest10591-21cut23:45:200
118999196,11cyclictest1177-21cyclictest22:20:140
118999195,11cyclictest15585-21cron19:25:010
118999194,12cyclictest0-21swapper/023:55:160
1189991915,2cyclictest0-21swapper/020:55:130
1189991911,5cyclictest15392-21usage_EthcTask00:05:310
118999191,14cyclictest5422-21memory22:30:220
118999191,14cyclictest5422-21memory22:30:220
118999191,14cyclictest30636-21kthreadcore20:45:210
118999191,14cyclictest19861-21ssh00:10:150
122599183,9cyclictest0-21swapper/323:50:153
122599182,9cyclictest0-21swapper/319:25:113
122599182,10cyclictest0-21swapper/320:30:143
122599182,10cyclictest0-21swapper/320:25:133
122599182,10cyclictest0-21swapper/319:35:133
1225991811,4cyclictest0-21swapper/322:30:133
1225991811,4cyclictest0-21swapper/322:30:133
122599181,12cyclictest0-21swapper/319:10:013
122599181,12cyclictest0-21swapper/319:10:013
122599181,11cyclictest0-21swapper/319:10:173
121399181,9cyclictest1177-21cyclictest19:10:012
120499187,8cyclictest345-21apt-get20:50:141
120499187,8cyclictest345-21apt-get20:50:131
120499181,9cyclictest1242-21latency_hist19:10:011
120499181,9cyclictest1242-21latency_hist19:10:011
120499181,8cyclictest21212-21kthreadcore21:10:191
120499181,15cyclictest24785-21kthreadcore20:05:221
1204991811,5cyclictest20388-21sshd19:28:251
120499181,14cyclictest17849-21tr22:55:251
1204991811,4cyclictest14476-21netstat22:35:251
120499181,14cyclictest11143-21cut19:20:151
120499181,13cyclictest3605-21sh23:25:131
120499181,13cyclictest1893-21ssh22:30:121
120499181,13cyclictest1893-21ssh22:30:121
118999188,7cyclictest5396-21sh21:20:140
118999188,7cyclictest17505-1kworker/u9:223:20:170
118999187,8cyclictest17972-21apt-get21:45:120
118999182,2cyclictest131rcu_preempt23:02:470
118999181,7cyclictest31330-21awk20:15:020
118999181,7cyclictest18058-21kthreadcore23:50:220
1189991811,5cyclictest16872-21sshd19:59:250
122599179,6cyclictest0-21swapper/323:26:373
120499178,6cyclictest9837-21latency_hist21:00:011
120499171,13cyclictest18437-21sed21:45:171
120499170,10cyclictest8571-21ls20:25:001
118999173,11cyclictest9440-21kernelversion20:25:180
118999172,12cyclictest1572-21fschecks_time19:10:160
118999171,14cyclictest20847-21timerandwakeup20:00:280
118999171,13cyclictest457-21cut20:50:170
118999171,13cyclictest457-21cut20:50:170
118999171,13cyclictest16264-21irqcore19:25:160
118999171,12cyclictest15187-21kthreadcore22:55:210
122599162,9cyclictest0-21swapper/320:45:143
122599162,9cyclictest0-21swapper/319:55:133
122599162,9cyclictest0-21swapper/319:50:043
120499168,5cyclictest0-21swapper/119:30:221
120499163,10cyclictest24829-21kthreadcore20:40:201
120499162,9cyclictest0-21swapper/120:15:151
120499162,10cyclictest0-21swapper/119:50:141
120499161,12cyclictest28884-21apt-get22:45:151
120499161,12cyclictest11454-21kthreadcore20:25:211
118999168,6cyclictest12-21ksoftirqd/019:10:010
118999168,6cyclictest12-21ksoftirqd/019:10:010
118999166,7cyclictest3167-21grep19:45:170
118999165,9cyclictest10771-21sendmail19:20:010
118999163,10cyclictest14365-21cut20:30:190
118999162,10cyclictest10188-21kthreadcore19:50:210
118999161,7cyclictest1721-21unixbench_singl19:40:280
118999161,13cyclictest25034-21kthreadcore20:05:230
118999161,12cyclictest15257-21cut21:05:180
122599159,4cyclictest0-21swapper/320:05:153
120499158,5cyclictest8020-21kthreadcore19:15:191
120499158,5cyclictest29506-21cat20:45:201
120499151,11cyclictest17733-21irqrtprio20:00:171
120499151,10cyclictest12499-21fschecks_time19:55:151
118999152,10cyclictest2002-21kthreadcore20:15:220
122599140,10cyclictest0-21swapper/321:00:133
122599122,5cyclictest0-21swapper/320:10:143
122599112,5cyclictest0-21swapper/320:15:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional