You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 13:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Jul 24, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13037996510,9cyclictest131rcu_preempt20:20:110
13037996420,34cyclictest131rcu_preempt20:50:120
1305499618,16cyclictest27-21ksoftirqd/200:20:122
1303799597,5cyclictest131rcu_preempt21:35:130
1304599561,46cyclictest3858-21ssh23:00:151
1304599551,41cyclictest17215-21ssh21:20:161
13067995345,5cyclictest35966irq/58-eth022:05:123
13045995213,31cyclictest21-21ksoftirqd/119:30:111
1303799521,45cyclictest24377-21ssh00:05:160
1306799502,28cyclictest35966irq/58-eth022:30:133
1306799502,28cyclictest35966irq/58-eth022:30:123
1305499502,36cyclictest28161-21ssh00:25:132
13037994830,9cyclictest12-21ksoftirqd/022:20:140
1303799481,9cyclictest131rcu_preempt22:40:140
1305499463,37cyclictest31564-21ssh21:10:142
1304599461,32cyclictest16778-21ssh23:25:131
1306799443,27cyclictest35966irq/58-eth022:00:123
1303799423,26cyclictest12985-21ssh23:05:150
1306799393,21cyclictest35966irq/58-eth021:15:163
1305499382,18cyclictest2554-21switchtime23:15:272
1305499382,18cyclictest2554-21switchtime23:15:262
1304599383,23cyclictest24789-21sh22:00:131
13067993721,11cyclictest35966irq/58-eth021:30:113
1305499372,18cyclictest11720-21ntp_192.168.11421:15:262
1303799375,25cyclictest12-21ksoftirqd/019:45:120
1303799375,15cyclictest12-21ksoftirqd/019:20:150
1304599363,22cyclictest31076-21ssh23:15:131
1304599363,22cyclictest31076-21ssh23:15:121
1304599361,29cyclictest28149-21apt-get00:25:121
1304599361,28cyclictest11573-21ssh23:40:141
1303799364,27cyclictest10453-21ssh22:10:120
1305499353,26cyclictest0-21swapper/221:32:022
1305499352,26cyclictest10082-21ssh00:15:162
1304599352,5cyclictest131rcu_preempt22:15:131
1305499349,19cyclictest27-21ksoftirqd/221:25:142
13067993322,8cyclictest35966irq/58-eth000:15:263
1305499332,25cyclictest6718-21ssh21:50:152
1304599331,23cyclictest0-21swapper/119:15:141
1306799327,15cyclictest35966irq/58-eth021:35:123
13067993216,9cyclictest35966irq/58-eth022:35:123
1305499322,19cyclictest10477-21ssh22:10:132
1305499315,17cyclictest27346irq/40-dwc2_hso22:20:142
1304599312,22cyclictest9099-21ssh22:45:141
13045993121,4cyclictest0-21swapper/121:50:141
13037993117,11cyclictest31357-21kworker/u8:022:05:130
1305499304,15cyclictest27346irq/40-dwc2_hso20:35:112
1305499303,19cyclictest0-21swapper/221:35:122
1305499303,18cyclictest27346irq/40-dwc2_hso21:40:142
1305499302,21cyclictest0-21swapper/220:30:132
1305499302,16cyclictest14446-21ssh22:30:142
1305499302,16cyclictest14446-21ssh22:30:132
1304599303,21cyclictest29668-21ssh23:50:151
1304599302,22cyclictest4016-21ssh21:47:201
1306799295,19cyclictest0-21swapper/321:45:143
1306799293,21cyclictest35966irq/58-eth021:10:153
1305499294,9cyclictest261rcuc/222:35:122
13054992921,5cyclictest2746ksoftirqd/222:40:122
1303799292,21cyclictest32564-21ssh21:27:200
13037992912,13cyclictest21020-21sh21:40:140
1306799282,15cyclictest35966irq/58-eth000:10:143
13067992818,6cyclictest35966irq/58-eth000:00:123
13067992818,6cyclictest0-21swapper/321:50:143
13067992810,12cyclictest0-21swapper/300:30:133
1305499285,17cyclictest27346irq/40-dwc2_hso20:50:132
1305499282,21cyclictest24076-21ssh23:28:252
1305499282,21cyclictest0-21swapper/223:14:272
1305499282,17cyclictest27346irq/40-dwc2_hso00:10:152
1305499282,10cyclictest21903-21ssh21:57:082
1303799282,7cyclictest131rcu_preempt19:25:140
1303799282,21cyclictest23454-21ssh22:35:130
1303799281,24cyclictest22065-21ssh23:10:140
1306799271,23cyclictest35966irq/58-eth019:45:123
1305499274,15cyclictest27346irq/40-dwc2_hso19:40:142
1305499272,20cyclictest9943-21/usr/sbin/munin23:55:262
13054992711,13cyclictest0-21swapper/223:00:002
1304599274,19cyclictest14447-21ssh22:30:141
1304599274,19cyclictest14447-21ssh22:30:131
1304599272,19cyclictest31789-21ssh21:26:241
1304599272,19cyclictest27494-21ssh21:42:251
1303799274,3cyclictest131rcu_preempt21:55:140
1303799272,19cyclictest31865-21ssh00:25:260
1303799272,19cyclictest27297-21ssh23:47:420
13037992720,3cyclictest12-21ksoftirqd/023:40:140
1303799271,20cyclictest15317-21ssh00:00:110
1303799271,19cyclictest10080-21rm00:15:160
1305499264,12cyclictest0-21swapper/222:05:142
1305499263,17cyclictest27346irq/40-dwc2_hso22:25:122
1305499261,9cyclictest25970-21latency_hist21:25:012
1304599262,19cyclictest9016-21ssh21:32:031
13045992612,10cyclictest21-21ksoftirqd/100:35:121
1304599261,18cyclictest2511-21ssh23:35:131
13045992610,11cyclictest21-21ksoftirqd/122:59:251
1303799264,4cyclictest131rcu_preempt23:30:140
1303799261,20cyclictest17458-21ssh00:35:270
1306799253,14cyclictest0-21swapper/321:26:243
1306799251,20cyclictest35966irq/58-eth020:55:143
1305499255,15cyclictest27346irq/40-dwc2_hso00:30:122
1305499252,18cyclictest27897-21ssh23:48:252
1305499252,18cyclictest24387-21ssh00:05:162
1303799252,20cyclictest19564-21ssh22:15:150
1303799251,22cyclictest4750-21ssh00:30:130
1306799243,16cyclictest35966irq/58-eth019:20:143
1306799243,13cyclictest0-21swapper/323:15:273
1306799243,13cyclictest0-21swapper/323:15:263
1305499248,7cyclictest17847-21latency_hist22:50:022
1305499242,7cyclictest0-21swapper/223:05:142
1305499242,17cyclictest0-21swapper/200:00:272
1305499241,4cyclictest0-21swapper/219:55:142
13054992412,10cyclictest32280-21ssh22:03:382
1305499241,18cyclictest11473-21ssh23:20:272
1304599242,17cyclictest22800-21apt-get19:20:141
13045992414,6cyclictest0-21swapper/123:30:111
1303799246,12cyclictest12-21ksoftirqd/022:50:140
1303799246,12cyclictest12-21ksoftirqd/022:50:130
1303799243,17cyclictest8123-21ssh21:15:160
1303799243,15cyclictest12-21ksoftirqd/000:20:120
1303799242,3cyclictest131rcu_preempt21:50:150
1303799241,4cyclictest131rcu_preempt20:40:140
13067992312,6cyclictest35966irq/58-eth022:45:123
1305499237,5cyclictest12511-21apt-key23:05:022
1305499234,8cyclictest27346irq/40-dwc2_hso20:20:112
1305499234,11cyclictest27346irq/40-dwc2_hso20:00:152
1305499232,6cyclictest0-21swapper/221:45:252
1305499231,19cyclictest19764-21grep19:50:192
1305499231,18cyclictest8578-21cat19:35:272
1305499231,18cyclictest31829-21kthreadcore23:50:232
1304599234,11cyclictest0-21swapper/120:15:121
1304599233,11cyclictest15433-21kworker/1:123:45:121
1303799239,4cyclictest131rcu_preempt21:20:190
13037992314,5cyclictest31043-21/usr/sbin/munin23:15:260
13037992314,5cyclictest31043-21/usr/sbin/munin23:15:250
13067992212,7cyclictest0-21swapper/323:10:173
1305499229,6cyclictest8597-21ssh23:36:592
1305499222,16cyclictest32191-21awk19:30:022
1305499222,15cyclictest27346irq/40-dwc2_hso19:45:112
1305499221,11cyclictest9457-21kthreadcore20:45:232
1305499221,11cyclictest21566-21basename21:00:002
1305499221,11cyclictest11171-21irqcore20:15:172
1305499221,10cyclictest21625-21kthreadcore22:15:232
1305499221,10cyclictest16093-21kthreadcore19:10:212
1304599229,3cyclictest131rcu_preempt22:05:261
13045992213,6cyclictest8687-21cron22:45:021
1304599221,18cyclictest20806-21ls20:25:171
1304599221,18cyclictest15411-21ls20:50:221
1304599221,18cyclictest1446-21cut20:05:171
1303799223,12cyclictest5408-21ssh22:25:130
13037992212,5cyclictest18753-21kthreadcore23:25:210
1306799216,10cyclictest35966irq/58-eth021:59:243
1306799213,11cyclictest0-21swapper/323:05:063
13067992110,9cyclictest0-21swapper/300:22:493
1305499212,4cyclictest0-21swapper/220:05:132
1305499212,16cyclictest18083-21/usr/sbin/munin22:50:182
1305499212,16cyclictest18083-21/usr/sbin/munin22:50:172
1305499211,9cyclictest28436-21kthreadcore21:05:212
1305499211,9cyclictest19860-21kthreadcore19:15:212
1305499211,16cyclictest8959-21kthreadcore20:10:212
1305499211,16cyclictest12110-21irqcore23:40:192
1305499211,11cyclictest4795-21grep19:35:012
1305499211,11cyclictest29561-21ntpq23:30:262
1305499211,10cyclictest2684-21grep20:40:172
1305499211,10cyclictest23867-21kthreadcore19:20:212
1305499211,10cyclictest14630-21kthreadcore00:35:202
1305499210,18cyclictest20631-21ls20:25:132
1304599219,9cyclictest31160-21apt21:10:011
1303799218,7cyclictest12-21ksoftirqd/020:05:120
1303799212,5cyclictest32277-21irqstats21:10:200
1303799212,16cyclictest7785-21ssh23:20:150
13037992113,6cyclictest23829-21sshd19:54:250
1303799211,15cyclictest3859-21ssh23:00:150
1306799203,11cyclictest0-21swapper/320:20:103
13067992010,8cyclictest0-21swapper/323:30:593
13054992012,5cyclictest26341-21idleruntime-cro21:05:012
1304599207,4cyclictest21622-21latency_hist21:00:021
1304599202,15cyclictest26689-1kworker/u9:100:30:131
1304599202,13cyclictest14345-21ssh23:59:261
1304599202,12cyclictest18135-21ssh22:50:161
1304599202,12cyclictest18135-21ssh22:50:161
13045992013,4cyclictest537-21apt-get00:10:051
13045992011,7cyclictest529-21cron00:10:001
13045992011,6cyclictest7916-21grep23:20:161
13045992011,5cyclictest25667-21ntp_192.168.11423:10:261
1304599201,15cyclictest25340-21apt-get20:30:101
1303799203,14cyclictest0-21swapper/021:45:150
13037992012,5cyclictest29613-21/usr/sbin/munin23:50:150
13037992011,6cyclictest5499-21kthreadcore21:30:200
13037992011,6cyclictest1815-21kthreadcore00:10:210
13037992011,5cyclictest26778-21kthreadcore22:00:210
1303799201,14cyclictest25450-21uniq20:30:130
1306799193,11cyclictest0-21swapper/323:47:433
1306799193,11cyclictest0-21swapper/323:29:253
1306799192,13cyclictest0-21swapper/300:35:123
1306799192,11cyclictest0-21swapper/321:20:243
13067991912,5cyclictest35966irq/58-eth000:25:123
1306799191,13cyclictest0-21swapper/322:25:163
1306799191,13cyclictest0-21swapper/320:30:143
1306799191,12cyclictest0-21swapper/320:05:143
1306799191,12cyclictest0-21swapper/319:40:283
13067991910,6cyclictest35966irq/58-eth023:52:313
13067991910,6cyclictest0-21swapper/323:22:563
13067991910,6cyclictest0-21swapper/321:40:143
1304599196,9cyclictest0-21swapper/120:45:161
1304599193,9cyclictest0-21swapper/119:55:141
1304599192,12cyclictest23871-21ssh21:59:241
1304599191,8cyclictest21118-21kthreadcore19:50:211
13045991912,4cyclictest10181-21sh00:15:161
1304599191,15cyclictest26448-21latency_hist21:05:011
1303799193,13cyclictest10933-21kthreadcore19:40:200
1303799192,9cyclictest11222-21latency_hist23:40:010
1303799191,5cyclictest0-21swapper/021:00:140
13037991911,6cyclictest27865-21kthreadcore22:55:200
1306799189,6cyclictest0-21swapper/323:40:163
1306799189,6cyclictest0-21swapper/322:44:053
1306799188,8cyclictest0-21swapper/322:50:033
1306799188,8cyclictest0-21swapper/322:50:023
1306799188,6cyclictest0-21swapper/300:08:463
1306799182,11cyclictest0-21swapper/323:57:273
1306799182,10cyclictest0-21swapper/320:35:143
1306799182,10cyclictest0-21swapper/319:30:133
1304599189,6cyclictest20341-21cron20:25:011
1304599182,12cyclictest16025-21irqstats00:00:191
1304599181,9cyclictest9817-21timerwakeupswit20:10:271
1304599181,15cyclictest32341-21sed19:30:011
1304599181,14cyclictest4940-21apt-get22:25:051
13045991810,5cyclictest31778-21tr22:20:231
13045991810,5cyclictest0-21swapper/120:35:231
1303799188,8cyclictest30247-21df20:35:150
1303799188,7cyclictest0-21swapper/020:55:130
1303799183,3cyclictest131rcu_preempt19:15:250
1303799183,11cyclictest31162-21diskmemload23:55:140
1303799182,11cyclictest12728-21ssh22:45:250
1303799181,2cyclictest131rcu_preempt19:10:140
1303799181,14cyclictest18103-21sed22:30:260
1303799181,14cyclictest18103-21sed22:30:250
13037991810,6cyclictest25097-21sshd20:28:240
1306799179,6cyclictest0-21swapper/323:38:063
1306799179,6cyclictest0-21swapper/322:58:333
1306799179,6cyclictest0-21swapper/322:15:483
1306799179,4cyclictest0-21swapper/323:00:143
1306799172,9cyclictest0-21swapper/319:35:133
1306799172,10cyclictest0-21swapper/322:20:153
1306799172,10cyclictest0-21swapper/322:10:133
1306799172,10cyclictest0-21swapper/320:00:153
1306799172,10cyclictest0-21swapper/319:55:143
1304599179,5cyclictest0-21swapper/119:35:161
1304599173,11cyclictest0-21swapper/120:40:141
1304599172,12cyclictest844-21munin-node22:10:141
1304599171,4cyclictest0-21swapper/123:07:501
13045991711,4cyclictest23409-21acpi22:35:101
1304599171,13cyclictest13579-21irqcore19:10:181
1304599171,13cyclictest12047-21ssh21:35:151
1304599171,13cyclictest10844-21cut21:15:221
13045991710,4cyclictest0-21swapper/100:20:321
1303799179,6cyclictest28613-21cron20:00:000
1303799178,6cyclictest15446-21sendmail20:20:000
1303799171,4cyclictest0-21swapper/019:32:280
13037991711,4cyclictest0-21swapper/019:35:270
13037991710,5cyclictest10600-21sshd20:45:240
1306799161,9cyclictest0-21swapper/319:50:233
1304599161,8cyclictest14633-21fschecks_time19:45:161
1304599161,12cyclictest10363-21sort19:40:181
1304599161,11cyclictest32495-21wc20:00:271
13045991610,4cyclictest32324-21tail21:10:211
1303799164,9cyclictest1028-21cron20:05:010
1303799162,12cyclictest30367-21timerwakeupswit21:05:300
1303799162,10cyclictest0-21swapper/020:10:140
1306799152,10cyclictest0-21swapper/320:10:133
1306799151,9cyclictest0-21swapper/321:00:233
1306799151,11cyclictest0-21swapper/319:10:163
1304599151,10cyclictest13141-21latency_hist19:10:011
1306799149,3cyclictest0-21swapper/320:44:173
1306799149,3cyclictest0-21swapper/320:16:253
1306799142,7cyclictest0-21swapper/319:15:133
1306799141,6cyclictest0-21swapper/320:45:313
1306799132,5cyclictest0-21swapper/321:05:153
1306799131,10cyclictest0-21swapper/319:30:023
1305499131,9cyclictest12948-21latency_hist19:10:022
1303799131,10cyclictest13060-21grep19:10:010
1306799111,8cyclictest0-21swapper/320:25:163
1306799111,7cyclictest0-21swapper/320:50:163
1306799101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional