You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 04:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Jul 02, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25397997644,20cyclictest35966irq/58-eth022:40:133
2534999564,41cyclictest22979-21ssh00:15:120
2534999564,40cyclictest28109-21ssh00:00:110
25397995333,12cyclictest35966irq/58-eth000:00:123
2536499537,4cyclictest131rcu_preempt19:50:111
2536499531,46cyclictest10080-21ssh21:45:121
2536499521,42cyclictest30424-21sd-accept19:15:131
2534999511,40cyclictest9979-21ssh23:50:120
2536499504,3cyclictest131rcu_preempt21:55:111
25364994731,8cyclictest21-21ksoftirqd/122:40:111
2537599463,3cyclictest131rcu_preempt22:30:132
2536499461,6cyclictest131rcu_preempt00:20:131
2536499452,8cyclictest131rcu_preempt20:50:121
2539799444,31cyclictest35966irq/58-eth000:10:153
2537599441,25cyclictest10088-21sh21:45:142
2536499442,31cyclictest17940-21ssh22:25:131
2534999444,34cyclictest8701-21ssh00:25:140
2539799422,26cyclictest35966irq/58-eth021:15:153
2534999411,28cyclictest23042-21ssh22:10:110
2539799403,23cyclictest0-21swapper/300:30:133
2539799403,23cyclictest0-21swapper/300:30:123
2539799403,22cyclictest35966irq/58-eth021:30:133
25349994018,8cyclictest17732-21sh23:17:190
25349994017,7cyclictest11416-21diskmemload22:04:080
25349994016,9cyclictest24474-21ssh23:56:090
2537599394,26cyclictest17753-21ssh00:30:132
2537599394,26cyclictest17753-21ssh00:30:132
2537599393,28cyclictest11807-21ssh21:10:122
25349993920,13cyclictest0-21swapper/021:45:130
25397993822,8cyclictest35966irq/58-eth023:10:113
2537599382,18cyclictest879-21munin-node23:45:112
25349993818,7cyclictest11416-21diskmemload23:26:410
25349993817,7cyclictest11416-21diskmemload22:53:160
25349993817,7cyclictest11416-21diskmemload22:44:460
25349993817,7cyclictest11416-21diskmemload22:08:010
25349993815,8cyclictest6365-21ssh23:46:120
2537599373,23cyclictest16674-21ssh23:00:142
25375993732,2cyclictest261rcuc/221:25:132
25349993718,6cyclictest11416-21diskmemload23:23:320
25349993716,7cyclictest11416-21diskmemload00:23:560
25349993716,6cyclictest10325-21ssh00:06:160
25349993713,8cyclictest11416-21diskmemload21:55:440
25349993712,9cyclictest30483-21ssh00:35:270
2534999371,26cyclictest6169-21ssh23:30:120
2539799362,22cyclictest35966irq/58-eth021:35:143
25375993617,11cyclictest27-21ksoftirqd/222:15:122
25349993617,6cyclictest11416-21diskmemload22:55:460
25349993615,7cyclictest11183-21ssh22:38:400
2539799357,17cyclictest35966irq/58-eth023:30:123
2539799353,20cyclictest35966irq/58-eth020:20:113
2539799353,20cyclictest35966irq/58-eth020:20:113
2537599352,23cyclictest20449-21ssh23:20:142
2534999356,17cyclictest12-21ksoftirqd/020:40:130
25349993518,13cyclictest0-21swapper/020:00:120
25349993514,7cyclictest14478-21ssh21:29:200
25397993416,13cyclictest0-21swapper/322:50:143
2539799341,20cyclictest35966irq/58-eth023:35:143
25349993415,6cyclictest9808-21ssh23:13:420
25349993414,7cyclictest22983-21ssh22:25:390
2539799336,18cyclictest35966irq/58-eth022:55:113
2537599333,21cyclictest29866-21ssh21:20:122
2536499332,23cyclictest32197-21ssh22:15:121
25364993319,5cyclictest21-21ksoftirqd/121:30:131
2537599326,20cyclictest261rcuc/223:05:122
25349993222,8cyclictest23462-21netstat20:45:240
2539799313,20cyclictest0-21swapper/323:25:123
2539799311,24cyclictest35966irq/58-eth022:45:133
2537599318,19cyclictest630-21Async09:36:312
2537599312,16cyclictest4866-21ssh00:05:172
25349993122,7cyclictest31005-21ssh23:06:080
25349993120,8cyclictest1461-21ssh22:32:510
25349993119,9cyclictest27225-21ssh21:17:180
25349993117,12cyclictest0-21swapper/021:10:110
25349993113,7cyclictest5980-21diskmemload21:24:560
25349993112,10cyclictest24014-21sh23:03:340
25349993111,7cyclictest29916-21ssh21:36:170
25349993110,7cyclictest11416-21diskmemload22:49:360
2539799304,19cyclictest33-21ksoftirqd/323:55:133
2539799302,16cyclictest35966irq/58-eth023:05:133
25364993021,6cyclictest11416-21diskmemload22:20:141
25364993011,10cyclictest131rcu_preempt20:05:111
25349993020,8cyclictest10075-21sshd20:34:100
25349993018,9cyclictest15752-21ssh22:22:550
25349993015,13cyclictest0-21swapper/020:10:120
25349993012,6cyclictest11416-21diskmemload23:44:560
2534999300,9cyclictest15564-21df_inode21:30:140
2539799290,18cyclictest35966irq/58-eth021:20:113
2536499291,23cyclictest4867-21ssh22:00:121
25349992924,3cyclictest15219-21apt-get23:35:150
25349992922,4cyclictest10084-21apt-get20:35:050
25349992919,8cyclictest30531-21/usr/sbin/munin19:15:160
25349992918,8cyclictest7531-21/usr/sbin/munin19:55:220
25349992918,8cyclictest7531-21/usr/sbin/munin19:55:220
25349992913,7cyclictest13831-21ssh00:10:090
25349992910,6cyclictest5417-21ssh22:16:250
2537599285,17cyclictest1042-21ssh21:40:132
2537599282,18cyclictest28254-21ssh21:55:142
2537599281,17cyclictest13939-21sh00:10:142
2536499288,9cyclictest25339-21cyclictest20:30:111
2536499282,19cyclictest844-21munin-node23:45:101
25349992819,6cyclictest10217-21needreboot21:05:220
25349992818,8cyclictest8664-21ssh21:43:490
25349992818,7cyclictest410-21sshd20:24:100
25349992818,7cyclictest410-21sshd20:24:100
25349992817,8cyclictest715-21proc_pri20:55:270
25349992817,8cyclictest27623-21kthreadcore19:45:210
25349992817,8cyclictest27623-21kthreadcore19:45:200
25349992814,10cyclictest0-21swapper/019:20:120
2539799274,17cyclictest35966irq/58-eth020:05:123
2539799272,11cyclictest35966irq/58-eth020:40:123
25397992712,10cyclictest35966irq/58-eth021:50:133
2537599278,5cyclictest12144-21kworker/2:021:30:132
2537599275,20cyclictest1977-21cut00:20:212
2537599273,16cyclictest27346irq/40-dwc2_hso20:20:112
2537599273,16cyclictest27346irq/40-dwc2_hso20:20:112
2537599272,20cyclictest27-21ksoftirqd/222:50:142
2537599272,16cyclictest27346irq/40-dwc2_hso20:25:122
2537599272,15cyclictest27346irq/40-dwc2_hso00:15:122
2537599272,11cyclictest0-21swapper/219:15:142
2536499272,20cyclictest32339-21ssh21:39:101
25349992719,6cyclictest5439-21nfs_client21:00:220
25349992719,6cyclictest17621-21interrupts19:35:180
25349992718,7cyclictest29387-21users20:50:310
25349992717,8cyclictest4244-21find20:25:270
25349992716,8cyclictest23614-21ssh00:31:420
25349992716,8cyclictest23614-21ssh00:31:410
2534999270,14cyclictest19196-21ssh21:50:120
2539799265,17cyclictest0-21swapper/321:45:133
2539799262,20cyclictest35966irq/58-eth022:00:113
2537599263,16cyclictest27346irq/40-dwc2_hso21:00:132
2537599263,14cyclictest27346irq/40-dwc2_hso00:35:122
2537599262,19cyclictest0-21swapper/223:35:152
2537599261,18cyclictest0-21swapper/219:55:142
2537599261,18cyclictest0-21swapper/219:55:142
2537599261,16cyclictest20865-21ssh21:15:142
2537599261,14cyclictest930-21snmpd23:50:122
2536499262,19cyclictest6043-21ssh21:40:421
25349992616,8cyclictest17194-21missed_timers20:05:220
2534999261,10cyclictest0-21swapper/019:50:120
2539799257,12cyclictest0-21swapper/300:22:213
2539799253,15cyclictest0-21swapper/323:00:143
2539799253,14cyclictest0-21swapper/323:45:123
2539799251,20cyclictest0-21swapper/300:15:123
25397992510,10cyclictest0-21swapper/323:23:103
2537599253,12cyclictest27346irq/40-dwc2_hso19:10:122
2537599252,20cyclictest0-21swapper/220:15:152
2537599252,15cyclictest27346irq/40-dwc2_hso23:55:142
2537599252,11cyclictest3659-21ssh22:35:132
25375992513,9cyclictest0-21swapper/220:55:242
2537599251,11cyclictest13998-21ssh22:05:132
2534999256,8cyclictest0-21swapper/019:10:030
25349992516,7cyclictest23976-21irqrtprio20:15:190
2539799243,17cyclictest35966irq/58-eth023:40:123
2537599243,16cyclictest27346irq/40-dwc2_hso22:55:102
2537599243,16cyclictest0-21swapper/221:50:452
2537599242,7cyclictest24470-21ssh21:35:142
2537599241,8cyclictest12575-21cpuspeed19:30:132
2537599241,15cyclictest27346irq/40-dwc2_hso23:25:122
2537599241,11cyclictest8912-21sh22:20:132
2536499248,11cyclictest21-21ksoftirqd/121:50:451
2536499241,18cyclictest16689-21ssh23:00:141
25349992416,4cyclictest0-21swapper/019:40:120
25349992413,8cyclictest844-21munin-node19:30:100
2534999240,8cyclictest11037-21/usr/sbin/munin19:25:230
2539799233,16cyclictest0-21swapper/321:55:143
2537599232,8cyclictest261rcuc/200:00:132
2537599232,6cyclictest0-21swapper/223:18:122
2537599232,5cyclictest0-21swapper/220:00:122
2537599232,16cyclictest27346irq/40-dwc2_hso19:20:122
2537599232,16cyclictest0-21swapper/222:00:472
2537599232,13cyclictest27346irq/40-dwc2_hso19:40:112
2537599231,17cyclictest930-21snmpd22:10:112
2537599231,12cyclictest27830-21missed_timers23:40:222
2539799229,8cyclictest35966irq/58-eth022:05:133
2539799222,12cyclictest35966irq/58-eth022:15:123
2539799222,12cyclictest0-21swapper/319:40:133
2537599222,6cyclictest0-21swapper/223:14:112
2537599221,17cyclictest27346irq/40-dwc2_hso00:25:142
2537599221,11cyclictest22138-21grep22:25:302
2536499223,14cyclictest15227-21ssh23:35:131
25364992213,5cyclictest21423-1kworker/1:2H00:25:171
2536499221,18cyclictest4398-21ssh22:35:191
2536499221,16cyclictest2241-21ssh23:10:131
2537599219,10cyclictest27-21ksoftirqd/219:45:222
2537599219,10cyclictest27-21ksoftirqd/219:45:212
2537599217,7cyclictest31935-21df_inode19:50:152
2537599213,9cyclictest13986-21cut20:05:152
2537599212,16cyclictest19862-21latency_hist20:45:022
2537599212,16cyclictest12365-21latency_hist19:30:022
2537599211,6cyclictest0-21swapper/220:30:122
2537599211,11cyclictest7171-21grep21:05:172
2536499212,16cyclictest29479-21ssh23:25:111
25364992112,6cyclictest8790-21cron20:00:011
25364992112,6cyclictest8790-21cron20:00:001
2536499211,17cyclictest10491-21ls21:05:271
2539799205,9cyclictest0-21swapper/322:25:123
2539799203,10cyclictest0-21swapper/319:45:143
2539799203,10cyclictest0-21swapper/319:45:143
2539799202,13cyclictest0-21swapper/322:24:113
2539799202,13cyclictest0-21swapper/321:40:143
2539799202,10cyclictest35966irq/58-eth020:45:133
25397992012,5cyclictest35966irq/58-eth021:10:233
2539799201,12cyclictest35966irq/58-eth000:05:183
2537599203,13cyclictest12737-21ssh22:40:132
2537599202,9cyclictest25494-21latency_hist19:10:022
2537599202,9cyclictest22432-21kthreadcore20:45:212
2537599201,9cyclictest18843-21fschecks_time20:10:152
2537599201,10cyclictest1-21systemd19:35:192
25375992010,8cyclictest26387-21usage_EthcTask22:45:322
2536499205,3cyclictest131rcu_preempt00:05:021
2536499202,13cyclictest31007-21df_inode22:50:141
2536499201,9cyclictest29488-21munin-run21:20:001
2536499201,7cyclictest21881-21kthreadcore23:55:221
2536499201,16cyclictest28324-21find20:20:111
2536499201,16cyclictest28324-21find20:20:101
25364992011,6cyclictest14341-21sshd21:29:091
2536499200,17cyclictest14275-21ls22:05:161
2539799199,3cyclictest0-21swapper/322:31:403
2539799193,11cyclictest0-21swapper/323:19:123
2539799193,11cyclictest0-21swapper/322:35:103
2539799192,10cyclictest0-21swapper/319:10:133
25397991913,4cyclictest0-21swapper/300:25:123
2539799191,13cyclictest0-21swapper/323:50:123
2539799191,11cyclictest0-21swapper/319:20:143
2537599191,9cyclictest25075-21tune2fs20:50:162
2537599191,8cyclictest11071-21kthreadcore20:35:182
2536499192,13cyclictest25259-21apt-get19:10:031
2536499192,12cyclictest21-21ksoftirqd/100:33:451
2536499192,12cyclictest21-21ksoftirqd/100:33:441
2536499191,7cyclictest0-21swapper/123:06:111
25364991914,3cyclictest9723-21ssh22:55:211
2536499191,14cyclictest24272-21ssh23:40:131
2536499191,12cyclictest15263-21ssh21:10:221
2536499191,10cyclictest1095-21ntpq21:20:241
2539799189,5cyclictest35966irq/58-eth021:25:143
2539799182,11cyclictest0-21swapper/322:10:133
2539799182,10cyclictest0-21swapper/320:50:133
2539799182,10cyclictest0-21swapper/320:35:053
2539799182,10cyclictest0-21swapper/320:15:153
2539799182,10cyclictest0-21swapper/319:25:053
2536499187,3cyclictest131rcu_preempt00:20:011
2536499184,4cyclictest0-21swapper/122:10:291
2536499184,11cyclictest23238-21cron20:15:011
2536499181,8cyclictest25713-21mailstats22:45:291
2536499181,14cyclictest2947-21cut19:20:151
2536499181,14cyclictest10723-21kthreadcore23:50:191
25364991810,6cyclictest11500-21fschecks_count23:15:151
2539799179,6cyclictest0-21swapper/300:36:513
2539799172,9cyclictest0-21swapper/320:25:143
2539799172,10cyclictest0-21swapper/320:55:143
2539799172,10cyclictest0-21swapper/320:00:143
2539799172,10cyclictest0-21swapper/319:15:143
25397991711,4cyclictest33-21ksoftirqd/319:50:163
25397991711,4cyclictest33-21ksoftirqd/319:30:173
2539799171,13cyclictest0-21swapper/319:35:163
2539799171,11cyclictest0-21swapper/321:10:023
2539799171,11cyclictest0-21swapper/319:55:133
2539799171,11cyclictest0-21swapper/319:55:133
2536499179,5cyclictest22691-21kthreadcore23:20:221
2536499179,5cyclictest21324-21kthreadcore20:45:201
2536499175,10cyclictest27566-21ssh22:30:181
2536499172,12cyclictest27191-21threads20:15:281
2536499172,11cyclictest14348-21irqcore00:10:191
2536499172,10cyclictest0-21swapper/120:35:141
2536499171,7cyclictest6388-21fschecks_time23:30:161
2536499171,12cyclictest28807-21irqstats00:00:181
2536499163,10cyclictest8750-21kthreadcore19:25:201
2536499163,10cyclictest30652-21tr20:55:201
2536499163,10cyclictest26916-21ssh00:35:141
2536499162,11cyclictest0-21swapper/119:35:121
2536499161,12cyclictest4860-21cut21:00:211
2536499161,12cyclictest25371-21kthreadcore19:40:221
2536499161,12cyclictest11698-21tr20:00:221
2534999161,12cyclictest25350-21latency_hist19:10:020
2539799153,6cyclictest0-21swapper/320:10:123
2536499156,6cyclictest0-21swapper/119:30:121
2536499154,8cyclictest28957-21kthreadcore19:45:211
2536499154,8cyclictest28957-21kthreadcore19:45:201
2536499151,11cyclictest15548-21irqcore20:40:181
2536499151,11cyclictest1383-21kthreadcore20:25:191
2536499142,10cyclictest25339-21cyclictest19:10:021
2539799131,10cyclictest0-21swapper/321:00:213
2539799122,7cyclictest0-21swapper/319:10:023
2539799122,6cyclictest0-21swapper/320:30:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional