You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 05:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon May 04, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2852995934,8cyclictest21-21ksoftirqd/121:05:141
286399583,29cyclictest9351-21ssh23:45:132
2852995417,26cyclictest21-21ksoftirqd/122:10:141
284399541,40cyclictest15974-21ssh21:25:110
2863995121,19cyclictest630-21Async09:36:312
286399511,11cyclictest0-21swapper/220:50:142
284399491,38cyclictest27439-21ssh23:55:130
287699485,22cyclictest35966irq/58-eth023:55:143
2852994721,17cyclictest21-21ksoftirqd/100:20:131
287699464,28cyclictest35966irq/58-eth021:50:133
2876994620,15cyclictest35966irq/58-eth020:35:153
284399464,39cyclictest21527-21ssh22:40:150
284399461,33cyclictest4160-21ssh00:00:130
2876994318,20cyclictest35966irq/58-eth022:15:113
286399431,27cyclictest1449-21ssh21:35:152
285299435,6cyclictest131rcu_preempt23:50:121
285299433,5cyclictest131rcu_preempt19:15:121
285299422,3cyclictest131rcu_preempt21:40:121
286399417,30cyclictest2882-21unixbench_multi00:15:282
286399415,9cyclictest23467-21if_err_eth022:05:182
285299416,26cyclictest2743-21ssh00:35:131
284399411,14cyclictest12-21ksoftirqd/022:20:100
287699402,25cyclictest35966irq/58-eth022:20:133
286399401,12cyclictest17729-21ssh22:20:122
2843994012,20cyclictest12-21ksoftirqd/020:40:110
286399393,27cyclictest7876-21ssh00:20:122
286399391,23cyclictest6842-21sshd20:55:112
286399391,23cyclictest6842-21sshd20:55:112
286399391,15cyclictest16850-21kthreadcore23:30:202
286399391,13cyclictest18430-21tr22:55:202
285299398,5cyclictest131rcu_preempt23:25:121
284399391,33cyclictest31221-21ssh00:15:120
286399386,9cyclictest12325-21latency_hist19:20:012
286399384,10cyclictest27346irq/40-dwc2_hso20:20:122
286399382,28cyclictest28359-21sh21:50:122
286399381,9cyclictest777-21sendmail_mailqu20:45:272
286399381,9cyclictest26261-21grep00:30:172
286399381,9cyclictest22450-21cut21:10:192
286399381,8cyclictest21469-21usage_EthcTask00:25:302
286399381,10cyclictest29058-21grep21:30:312
2843993821,11cyclictest2339-21sh21:52:190
286399371,9cyclictest7194-21kthreadcore21:55:212
286399371,9cyclictest7194-21kthreadcore21:55:202
286399371,14cyclictest5323-21kthreadcore19:10:202
286399371,14cyclictest31829-21sort19:40:142
286399371,14cyclictest12676-21fschecks_time22:35:172
285299371,33cyclictest5071-21ssh21:55:131
285299371,33cyclictest5071-21ssh21:55:121
284399371,28cyclictest3434-21ssh22:30:140
287699361,21cyclictest35966irq/58-eth022:55:123
286399368,3cyclictest4868-21kthreadcore22:30:202
286399363,9cyclictest14149-21awk21:40:282
286399361,7cyclictest14514-21ntp_kernel_pll_23:10:252
284399364,22cyclictest23697-21ssh23:35:150
287699352,17cyclictest35966irq/58-eth000:35:143
287699351,20cyclictest35966irq/58-eth022:05:123
286399351,9cyclictest4368-21ssh23:40:302
286399351,9cyclictest21883-21ssh23:50:242
286399351,5cyclictest19745-21irqcore21:45:172
2863993515,17cyclictest22079-21ssh22:40:192
286399343,6cyclictest15941-21ssh21:25:102
286399341,9cyclictest7719-21irqcore22:50:182
286399341,7cyclictest9063-21sort19:50:152
286399341,7cyclictest27586-21fschecks_count23:55:162
286399341,6cyclictest14996-21latency_hist20:30:022
286399341,13cyclictest16192-21idleruntime-cro21:05:002
286399341,11cyclictest29563-21kthreadcore19:35:222
286399341,10cyclictest10078-21kthreadcore22:15:202
285299342,24cyclictest0-21swapper/120:35:141
284399349,20cyclictest12-21ksoftirqd/019:40:140
284399344,9cyclictest12-21ksoftirqd/023:20:120
2843993413,16cyclictest12-21ksoftirqd/019:30:120
287699339,6cyclictest0-21swapper/321:15:023
286399334,21cyclictest1849-21ssh23:05:142
286399333,9cyclictest7911-21grep00:00:272
286399332,26cyclictest0-21swapper/222:13:112
284399332,19cyclictest16204-21ssh22:55:120
286399325,15cyclictest27346irq/40-dwc2_hso23:20:112
286399321,11cyclictest4260-21find20:15:282
286399321,11cyclictest4260-21find20:15:272
286399321,10cyclictest9691-21unixbench_multi23:25:292
286399320,9cyclictest22133-21kthreadcore20:35:212
287699313,21cyclictest35966irq/58-eth022:00:123
286399316,5cyclictest27027-21cat00:10:322
286399311,8cyclictest26713-21/usr/sbin/munin22:25:292
286399311,6cyclictest19912-21kthreadcore19:25:202
286399311,12cyclictest10806-21grep21:20:272
285299312,24cyclictest12485-21ssh22:35:151
285299312,22cyclictest25968-21ssh00:30:131
285299312,19cyclictest1428-21ssh21:35:151
284399312,21cyclictest23118-21ssh22:05:110
287699303,16cyclictest35966irq/58-eth022:45:123
286399302,10cyclictest28215-21/usr/sbin/munin20:10:122
286399301,12cyclictest31020-21irqcore22:45:182
285299302,23cyclictest0-21swapper/123:00:131
287699292,16cyclictest35966irq/58-eth021:40:123
286399292,25cyclictest0-21swapper/223:38:112
286399292,10cyclictest16992-21irqrtprio21:05:212
286399291,6cyclictest24938-21ssh23:15:492
286399291,10cyclictest12685-21fschecks_count19:20:152
286399290,11cyclictest18709-21fschecks_count20:00:152
285299291,20cyclictest10809-21ssh22:50:241
284399292,21cyclictest21001-21diskmemload22:50:130
287699284,12cyclictest35966irq/58-eth000:25:113
287699283,16cyclictest0-21swapper/322:54:113
285299281,17cyclictest27441-21kworker/u8:120:05:101
287699275,15cyclictest0-21swapper/323:45:143
287699273,17cyclictest0-21swapper/322:12:233
2876992710,11cyclictest35966irq/58-eth023:35:153
286399272,20cyclictest0-21swapper/219:55:142
286399272,13cyclictest14093-21ssh22:00:122
285299274,11cyclictest0-21swapper/119:20:121
285299271,20cyclictest9360-21ssh23:45:131
284399277,3cyclictest131rcu_preempt00:20:130
284399272,19cyclictest18924-21ssh22:37:280
287699263,17cyclictest35966irq/58-eth000:10:143
287699262,17cyclictest0-21swapper/321:57:203
287699262,17cyclictest0-21swapper/321:57:203
2876992614,8cyclictest35966irq/58-eth000:00:133
286399261,11cyclictest21001-21diskmemload23:04:242
285299262,19cyclictest21-21ksoftirqd/121:45:141
2852992610,3cyclictest21-21ksoftirqd/121:10:121
284399262,20cyclictest14628-21ssh23:30:130
284399262,18cyclictest6119-21ssh22:12:220
2843992612,8cyclictest0-21swapper/023:27:450
284399261,18cyclictest26769-21ssh22:25:120
287699253,13cyclictest0-21swapper/323:10:133
287699252,17cyclictest0-21swapper/323:23:103
287699251,17cyclictest0-21swapper/300:05:143
285299259,11cyclictest1833-21ssh23:05:141
284399257,3cyclictest131rcu_preempt00:30:120
284399251,18cyclictest25227-21ssh23:00:130
2876992410,9cyclictest35966irq/58-eth000:20:133
286399242,13cyclictest27346irq/40-dwc2_hso19:45:122
2852992416,5cyclictest0-21swapper/119:35:201
2852992413,8cyclictest19818-21cron20:35:011
2852992413,8cyclictest19818-21cron20:35:001
2852992413,7cyclictest22249-21sh00:10:151
284399248,5cyclictest12-21ksoftirqd/021:30:300
284399247,4cyclictest131rcu_preempt21:40:120
284399246,13cyclictest10884-21diskmemload23:10:130
2843992415,6cyclictest12-21ksoftirqd/020:50:130
284399241,3cyclictest131rcu_preempt23:05:130
2843992412,10cyclictest12-21ksoftirqd/023:47:410
285299232,16cyclictest3899-21ssh21:54:101
285299232,14cyclictest15962-21apt-get21:25:121
285299232,14cyclictest14054-21ssh22:00:121
285299231,10cyclictest21731-21cat22:20:281
284399239,2cyclictest131rcu_preempt00:25:120
284399234,13cyclictest3689-1kworker/u9:221:20:120
284399232,6cyclictest11378-21munin-run00:40:010
284399232,16cyclictest22078-21ssh23:50:260
284399232,15cyclictest19884-21ssh23:15:160
284399231,19cyclictest27045-21ls19:35:160
284399231,18cyclictest17567-21uniq19:25:160
284399231,16cyclictest12-21ksoftirqd/021:15:110
287699223,12cyclictest0-21swapper/323:05:153
287699223,12cyclictest0-21swapper/321:25:123
2876992212,7cyclictest35966irq/58-eth000:15:123
286399222,14cyclictest0-21swapper/200:35:152
286399221,9cyclictest930-21snmpd20:30:142
286399221,9cyclictest930-21snmpd20:30:132
285299225,7cyclictest32207-21kthreadcore21:15:201
285299221,7cyclictest19980-21df_inode23:15:161
2852992212,7cyclictest32558-21sh22:45:221
284399228,11cyclictest13256-21cat22:15:290
284399225,5cyclictest12-21ksoftirqd/023:40:130
284399221,17cyclictest13196-21sh00:05:150
284399221,15cyclictest16324-21ls19:20:270
287699219,5cyclictest0-21swapper/321:30:203
2876992111,7cyclictest0-21swapper/321:35:233
286399216,7cyclictest27-21ksoftirqd/219:30:192
286399213,9cyclictest25056-21fschecks_time20:40:152
286399211,11cyclictest31094-21sh21:15:192
286399210,11cyclictest23652-21fschecks_time20:05:152
285299215,6cyclictest2829-21cyclictest23:40:141
285299213,12cyclictest0-21swapper/123:30:141
285299212,15cyclictest21001-21diskmemload23:10:251
285299212,14cyclictest27410-21ntp_states23:35:271
2852992113,5cyclictest25038-21cut20:40:141
2852992111,7cyclictest16187-21cron21:05:001
285299211,16cyclictest30819-21timerwakeupswit22:25:281
285299211,16cyclictest17490-21fschecks_count19:25:151
284399212,13cyclictest12-21ksoftirqd/019:15:120
2843992115,4cyclictest0-21swapper/021:55:120
2843992115,4cyclictest0-21swapper/021:55:120
2843992114,4cyclictest12-21ksoftirqd/020:00:250
287699208,9cyclictest0-21swapper/323:51:003
2876992011,6cyclictest35966irq/58-eth021:20:123
286399201,9cyclictest2906-21latency_hist19:10:022
285299203,12cyclictest8713-21ssh22:15:111
285299201,9cyclictest29628-21ps23:20:181
2852992011,6cyclictest614-21dbus-daemon20:12:041
285299200,17cyclictest25488-21ls19:30:231
284399209,3cyclictest12-21ksoftirqd/022:45:130
284399209,3cyclictest12-21ksoftirqd/021:35:150
284399209,2cyclictest12-21ksoftirqd/020:20:240
284399202,14cyclictest19354-21ssh21:45:130
284399201,3cyclictest131rcu_preempt21:05:260
284399201,16cyclictest11724-21df21:00:140
284399201,13cyclictest20211-21cut20:35:160
284399200,17cyclictest18605-21ls20:30:230
284399200,17cyclictest18605-21ls20:30:230
284399200,16cyclictest7678-21ls19:15:010
284399200,15cyclictest2002-21ls20:50:020
287699195,7cyclictest35966irq/58-eth023:00:123
2876991911,5cyclictest0-21swapper/322:30:233
285299198,8cyclictest7126-21ntp_states22:30:261
285299197,3cyclictest131rcu_preempt00:25:141
285299191,8cyclictest17902-21/usr/sbin/munin00:05:311
2852991911,5cyclictest4902-21kthreadcore00:00:201
2852991911,4cyclictest27670-21usage_EthcTask22:05:301
285299191,10cyclictest25011-21memory22:40:241
284399199,2cyclictest12-21ksoftirqd/022:00:260
284399197,3cyclictest0-21swapper/000:14:010
2843991913,4cyclictest0-21swapper/021:10:120
284399191,16cyclictest15222-21ls20:30:030
287699189,6cyclictest0-21swapper/321:19:213
287699183,10cyclictest0-21swapper/321:48:113
287699182,10cyclictest0-21swapper/322:35:113
287699182,10cyclictest0-21swapper/321:00:133
287699182,10cyclictest0-21swapper/320:40:123
287699182,10cyclictest0-21swapper/320:30:043
287699182,10cyclictest0-21swapper/320:30:043
287699182,10cyclictest0-21swapper/320:25:133
287699182,10cyclictest0-21swapper/320:05:043
287699182,10cyclictest0-21swapper/320:00:153
287699182,10cyclictest0-21swapper/319:30:133
287699181,9cyclictest0-21swapper/323:30:103
287699181,13cyclictest0-21swapper/320:50:153
2876991810,4cyclictest0-21swapper/323:29:083
285299189,7cyclictest20148-21wc22:55:291
285299186,4cyclictest21-21ksoftirqd/120:20:191
285299185,11cyclictest24867-21/usr/sbin/munin21:30:241
285299181,6cyclictest32280-21irqrtprio19:40:191
285299181,14cyclictest12754-21grep19:50:291
285299181,14cyclictest11461-21ssh21:20:291
285299181,13cyclictest27559-21sh23:55:151
285299181,12cyclictest0-21swapper/100:18:211
285299180,12cyclictest5292-21latency_hist20:20:021
285299180,12cyclictest5292-21latency_hist20:20:011
284399182,11cyclictest0-21swapper/020:55:140
284399182,11cyclictest0-21swapper/020:55:130
2843991812,4cyclictest0-21swapper/020:05:200
284399181,15cyclictest12683-21ls19:50:270
2843991811,4cyclictest12-21ksoftirqd/020:10:250
287699179,6cyclictest0-21swapper/323:15:323
287699179,6cyclictest0-21swapper/300:31:393
287699178,6cyclictest0-21swapper/322:40:083
287699171,9cyclictest0-21swapper/319:55:133
287699171,12cyclictest35966irq/58-eth023:40:143
287699171,11cyclictest35966irq/58-eth020:45:143
285299172,11cyclictest23142-21apt-get20:05:021
285299171,9cyclictest701-21ntp_states20:45:261
2852991711,3cyclictest21-21ksoftirqd/119:10:201
285299171,12cyclictest12662-21kthreadcore20:25:191
284399173,11cyclictest5291-21idleruntime-cro20:20:010
284399173,11cyclictest5291-21idleruntime-cro20:20:000
284399172,10cyclictest0-21swapper/019:45:140
284399171,6cyclictest0-21swapper/019:55:240
287699163,6cyclictest0-21swapper/319:20:123
287699162,9cyclictest0-21swapper/320:20:143
287699161,9cyclictest0-21swapper/320:15:013
287699161,7cyclictest0-21swapper/319:15:123
285299164,9cyclictest15857-21kthreadcore19:55:201
285299163,10cyclictest4667-21cut20:50:211
285299161,12cyclictest7030-21fschecks_time20:55:151
285299161,12cyclictest7030-21fschecks_time20:55:151
285299161,12cyclictest4509-21sed19:45:171
284399162,11cyclictest2907-21wc19:10:020
287699152,10cyclictest0-21swapper/319:35:153
287699151,7cyclictest0-21swapper/322:25:193
2876991510,3cyclictest0-21swapper/319:40:283
287699142,10cyclictest0-21swapper/319:45:133
287699141,10cyclictest0-21swapper/320:15:193
287699141,10cyclictest0-21swapper/320:15:193
285299141,10cyclictest2867-21latency_hist19:10:031
287699132,6cyclictest0-21swapper/319:50:123
287699131,9cyclictest0-21swapper/321:10:023
287699131,8cyclictest0-21swapper/320:55:123
287699131,8cyclictest0-21swapper/320:55:113
287699122,6cyclictest0-21swapper/319:25:143
287699121,9cyclictest0-21swapper/319:10:153
28769991,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional