You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-06 - 23:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Tue Jan 06, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2994699616,5cyclictest131rcu_preempt07:20:130
2994699583,47cyclictest14874-21ssh11:55:150
2996699551,15cyclictest0-21swapper/207:25:132
29977995342,8cyclictest28366irq/58-eth010:30:133
29966995339,9cyclictest2253-21apt-get08:55:122
2995799532,40cyclictest19565-21ssh10:25:131
29966995238,10cyclictest22235-21apt-get10:45:132
2996699512,29cyclictest9904-21ssh11:15:112
2996699501,18cyclictest0-21swapper/209:55:122
29966994822,16cyclictest16525-21diskmemload09:15:122
2994699483,9cyclictest131rcu_preempt07:25:130
2994699481,7cyclictest131rcu_preempt09:55:110
29966994734,4cyclictest16905-21apt-get09:10:142
29966994627,8cyclictest26092-21ssh12:20:152
2994699463,5cyclictest131rcu_preempt07:30:100
29966994528,14cyclictest1169-21ssh11:10:112
29966994524,15cyclictest2355-21ssh12:25:132
29966994516,21cyclictest13612-21ssh10:57:162
29966994516,21cyclictest13612-21ssh10:57:162
2995799452,28cyclictest134-21systemd-journal12:00:131
29957994514,25cyclictest21-21ksoftirqd/111:25:141
2994699452,27cyclictest3664-21ssh11:30:130
29966994429,4cyclictest27324-21ssh11:25:142
2995799447,23cyclictest2370-21ssh12:25:141
2997799434,25cyclictest28366irq/58-eth011:05:113
29966994335,6cyclictest10928-21apt-get09:25:142
29966994322,18cyclictest0-21swapper/209:05:112
29977994217,13cyclictest28366irq/58-eth012:05:133
29966994228,9cyclictest23612-21apt-get12:00:142
2995799423,34cyclictest8572-21ssh12:10:131
29946994225,11cyclictest0-21swapper/011:15:120
2994699422,29cyclictest24871-21ssh11:05:100
29966994131,5cyclictest21215-21apt-get11:40:142
29966994115,9cyclictest11563-21rm11:51:162
29966994114,18cyclictest25490-21ssh09:31:322
29966994114,14cyclictest4546-21sh10:35:032
29966994112,12cyclictest19489-21ssh10:25:102
2995799414,30cyclictest21229-21ssh11:40:141
2997799401,26cyclictest28366irq/58-eth010:20:133
29966994033,3cyclictest2496-21apt-get07:15:132
29966994029,8cyclictest30113-21apt-get07:10:132
29966994021,17cyclictest0-21swapper/208:35:122
29966994015,9cyclictest24037-21ssh12:17:532
2994699401,29cyclictest8110-21ssh10:00:140
2997799399,16cyclictest28366irq/58-eth011:40:133
29977993919,16cyclictest0-21swapper/310:25:133
29977993917,15cyclictest28366irq/58-eth007:40:103
29966993931,6cyclictest25046-21apt-get08:45:142
29966993929,7cyclictest8562-21apt-get12:10:122
29966993921,15cyclictest16525-21diskmemload12:30:142
29966993918,18cyclictest0-21swapper/207:30:102
29966993917,18cyclictest0-21swapper/207:20:142
29966993912,12cyclictest0-21swapper/212:10:012
2996699391,18cyclictest24882-21ssh11:05:112
2994699392,4cyclictest131rcu_preempt10:55:150
2994699392,4cyclictest131rcu_preempt10:55:150
29966993830,4cyclictest10556-21apt-get08:30:132
29966993819,14cyclictest0-21swapper/210:50:132
2997799372,23cyclictest28366irq/58-eth010:10:143
2997799372,23cyclictest28366irq/58-eth010:10:133
29966993723,11cyclictest0-21swapper/208:20:122
2996699372,28cyclictest23224-21cat12:35:222
29966993713,9cyclictest24934-21ssh11:22:242
29966993713,7cyclictest0-21swapper/210:01:262
29966993711,17cyclictest0-21swapper/211:01:482
29966993711,11cyclictest0-21swapper/211:38:142
29966993710,11cyclictest0-21swapper/211:30:592
29957993731,3cyclictest21-21ksoftirqd/107:15:121
2997799363,18cyclictest28366irq/58-eth010:05:143
2996699367,12cyclictest0-21swapper/210:30:092
29966993619,7cyclictest1005-21/usr/sbin/munin11:45:232
29966993619,7cyclictest1005-21/usr/sbin/munin11:45:222
29966993618,9cyclictest22138-21ntp_states08:05:252
29966993618,15cyclictest14416-21kthreadcore08:00:182
29966993616,16cyclictest0-21swapper/211:55:152
29966993613,7cyclictest0-21swapper/209:47:262
2995799369,5cyclictest131rcu_preempt12:20:141
2994699362,31cyclictest16947-21ssh10:05:130
29966993521,11cyclictest17130-21sendmail-mta10:22:222
29966993520,9cyclictest25711-21apt-get10:10:132
29966993520,9cyclictest25711-21apt-get10:10:122
29966993520,12cyclictest7572-21ntp_states07:50:262
29966993519,14cyclictest0-21swapper/209:40:122
29966993511,15cyclictest0-21swapper/210:17:562
29966993511,11cyclictest19417-21ssh10:41:542
2994699352,21cyclictest16916-21ssh09:10:120
29977993425,5cyclictest321rcuc/309:25:133
2996699349,12cyclictest28751-21apt-get09:35:142
29966993418,13cyclictest25737-21tail07:35:282
29966993417,8cyclictest11108-21cat09:00:272
2994699343,23cyclictest17261-21ssh12:15:110
29977993323,7cyclictest0-21swapper/310:35:133
29966993322,8cyclictest0-21swapper/208:50:122
29966993320,9cyclictest0-21swapper/208:15:112
29966993317,14cyclictest29625-21rm09:52:552
29957993320,3cyclictest21-21ksoftirqd/111:10:111
2994699335,18cyclictest630-21tcf-agent07:45:120
2997799321,15cyclictest28366irq/58-eth012:35:143
29966993216,13cyclictest24287-21usage_Anybus08:40:312
29966993216,12cyclictest5645-21/usr/sbin/munin08:25:272
29966993212,11cyclictest16844-21sh10:05:072
29977993118,6cyclictest28366irq/58-eth010:50:133
2996699318,9cyclictest0-21swapper/209:22:572
29966993111,17cyclictest27169-21irqcore07:40:162
2994699313,21cyclictest5233-21ssh09:40:140
2994699312,20cyclictest16189-21rm11:00:150
29966993018,8cyclictest2645-21ntpq07:45:252
29966993015,11cyclictest0-21swapper/207:55:122
2995799308,4cyclictest131rcu_preempt11:05:121
2995799302,23cyclictest13479-21ssh10:40:131
29957993013,15cyclictest201rcuc/109:15:131
2994699302,21cyclictest4839-21ssh11:10:250
29946993015,9cyclictest29929-21cyclictest11:35:120
29966992916,9cyclictest23461-21/usr/sbin/munin08:10:282
29957992913,10cyclictest19455-21kworker/u8:110:30:131
2994699292,19cyclictest3069-21/usr/sbin/munin12:25:190
29946992918,8cyclictest10260-21kworker/u8:309:25:150
29946992916,7cyclictest0-21swapper/010:25:130
2997799282,14cyclictest28366irq/58-eth011:10:113
2995799283,20cyclictest4699-21ssh10:35:151
2994699283,19cyclictest8594-21sshd12:10:150
2994699283,18cyclictest10877-21ssh10:20:130
2994699282,19cyclictest13984-21grep12:30:210
2997799272,16cyclictest0-21swapper/312:30:223
2994699274,18cyclictest16525-21diskmemload09:20:120
2994699271,18cyclictest22843-21ssh09:50:120
2997799260,13cyclictest28366irq/58-eth010:40:113
29957992613,11cyclictest0-21swapper/111:52:371
2995799261,19cyclictest18678-21ssh11:20:141
29957992611,10cyclictest21-21ksoftirqd/110:15:131
2997799251,16cyclictest0-21swapper/312:25:193
2997799251,15cyclictest0-21swapper/312:19:583
2994699252,17cyclictest31220-21ssh12:20:570
2994699251,17cyclictest25726-21rm10:10:140
2994699251,17cyclictest25726-21rm10:10:140
29946992511,4cyclictest131rcu_preempt10:40:200
2997799244,10cyclictest0-21swapper/309:45:113
2995799242,16cyclictest19781-21ssh12:35:151
2995799242,14cyclictest22251-21ssh10:45:141
2995799241,17cyclictest28754-21ssh09:35:131
2994699241,11cyclictest27129-21users10:45:310
2997799233,13cyclictest0-21swapper/311:20:143
2995799235,15cyclictest0-21swapper/110:05:141
2995799231,19cyclictest16322-21uniq11:00:161
2995799231,17cyclictest14051-21ssh09:45:131
2995799231,15cyclictest31042-21ssh10:50:131
29946992313,6cyclictest28852-21cpuspeed09:35:150
2994699231,16cyclictest7155-21ls10:55:010
2994699231,16cyclictest18589-21kworker/u8:211:20:130
2997799223,13cyclictest0-21swapper/312:20:573
2995799229,11cyclictest11063-1kworker/u9:109:30:141
2995799228,5cyclictest21-21ksoftirqd/111:49:561
2995799228,5cyclictest21-21ksoftirqd/111:49:561
29957992213,6cyclictest0-21swapper/109:00:271
2994699225,9cyclictest12-21ksoftirqd/011:45:120
2994699225,9cyclictest12-21ksoftirqd/011:45:120
2994699223,15cyclictest16525-21diskmemload11:50:120
2994699221,9cyclictest28453-21df_inode10:30:150
29946992214,5cyclictest0-21swapper/008:45:170
29946992212,5cyclictest14050-21sh09:45:130
2994699221,18cyclictest29546-21ls09:15:280
2994699220,18cyclictest18891-21ls08:05:160
2997799216,12cyclictest0-21swapper/309:40:313
2997799213,13cyclictest0-21swapper/312:10:573
2997799211,13cyclictest0-21swapper/311:30:133
29966992112,7cyclictest29929-21cyclictest07:10:012
29966992112,7cyclictest29929-21cyclictest07:10:012
2995799219,3cyclictest131rcu_preempt09:40:131
2995799216,6cyclictest14891-21kthreadcore07:25:231
2995799212,11cyclictest10958-21diskmemload09:25:141
29957992115,4cyclictest0-21swapper/110:05:011
29957992113,5cyclictest16399-21timerwakeupswit11:35:281
29957992113,5cyclictest14840-21/usr/sbin/munin11:55:211
2995799211,2cyclictest131rcu_preempt10:56:291
2995799211,2cyclictest131rcu_preempt10:56:291
29957992112,5cyclictest32421-21kthreadcore09:55:191
29957992111,6cyclictest26316-21latency09:50:231
2994699213,5cyclictest28129-21sed12:40:010
2994699211,15cyclictest4078-21apt-get07:50:130
29946992110,9cyclictest0-21swapper/012:00:120
29946992110,3cyclictest131rcu_preempt11:40:220
2997799209,6cyclictest0-21swapper/312:00:583
29977992010,8cyclictest0-21swapper/310:16:543
29977992010,7cyclictest28366irq/58-eth009:55:123
2995799206,10cyclictest21-21ksoftirqd/109:05:121
2995799203,12cyclictest25725-21sh10:10:141
2995799203,12cyclictest25725-21sh10:10:141
2995799201,2cyclictest131rcu_preempt07:50:011
2995799201,15cyclictest17361-21df12:15:141
29957992011,5cyclictest10288-21irqcore11:15:171
2994699206,3cyclictest131rcu_preempt10:15:170
2994699202,12cyclictest19841-21ssh09:30:120
2994699201,13cyclictest1070-21idleruntime08:20:160
2994699201,10cyclictest6093-21grep08:55:280
2997799199,8cyclictest0-21swapper/311:26:593
2997799199,6cyclictest0-21swapper/309:33:423
2997799199,6cyclictest0-21swapper/309:21:493
2997799193,11cyclictest0-21swapper/311:54:573
2997799193,11cyclictest0-21swapper/311:38:573
29977991913,4cyclictest0-21swapper/311:00:253
2997799191,12cyclictest0-21swapper/309:20:033
2997799191,12cyclictest0-21swapper/308:45:163
2997799191,11cyclictest0-21swapper/308:55:283
2995799199,7cyclictest616-21dbus-daemon08:14:261
2995799193,14cyclictest32344-21ssh12:05:121
2995799192,6cyclictest134-21systemd-journal08:10:011
2995799192,13cyclictest10861-21apt-get10:20:131
2995799191,9cyclictest17613-21unixbench_singl08:00:271
2995799191,9cyclictest131rcu_preempt07:55:161
2995799191,8cyclictest27412-21kthreadcore07:40:181
29957991914,3cyclictest0-21swapper/111:30:151
2995799191,10cyclictest14924-21switchtime12:30:281
2995799190,3cyclictest131rcu_preempt09:20:331
2994699198,8cyclictest2017-21cron08:55:020
2994699198,8cyclictest16519-21cron09:10:000
2994699195,11cyclictest0-21swapper/009:00:120
2994699193,11cyclictest0-21swapper/012:05:130
2994699192,15cyclictest7395-21kthreadcore10:35:220
2994699191,10cyclictest21818-21tr08:40:210
2997799189,6cyclictest28366irq/58-eth010:00:133
2997799189,6cyclictest0-21swapper/309:36:573
2997799188,7cyclictest0-21swapper/311:45:303
2997799188,7cyclictest0-21swapper/311:45:293
2997799183,10cyclictest0-21swapper/311:55:123
2997799182,10cyclictest0-21swapper/309:05:133
2997799182,10cyclictest0-21swapper/308:50:153
2997799182,10cyclictest0-21swapper/307:50:143
2997799182,10cyclictest0-21swapper/307:45:133
2997799181,12cyclictest0-21swapper/311:15:143
2997799181,12cyclictest0-21swapper/310:50:023
2997799181,11cyclictest0-21swapper/308:05:213
2997799181,11cyclictest0-21swapper/307:35:283
2997799181,10cyclictest28366irq/58-eth008:35:123
2997799181,10cyclictest0-21swapper/308:05:023
2995799183,10cyclictest0-21swapper/108:45:141
29957991811,4cyclictest0-21swapper/107:50:281
2995799180,13cyclictest9402-21timerwakeupswit08:25:271
2994699187,8cyclictest27309-21apt-get11:25:130
2994699183,8cyclictest0-21swapper/008:00:110
2994699183,11cyclictest0-21swapper/007:10:130
2994699182,10cyclictest0-21swapper/008:10:140
29946991812,4cyclictest0-21swapper/008:30:220
2997799179,6cyclictest0-21swapper/310:55:073
2997799179,6cyclictest0-21swapper/310:55:063
2997799179,6cyclictest0-21swapper/309:13:503
2997799179,5cyclictest28366irq/58-eth009:50:133
2997799172,10cyclictest0-21swapper/309:00:123
2997799171,12cyclictest0-21swapper/307:20:173
2995799175,9cyclictest24300-21cut07:35:211
2995799173,11cyclictest18726-21kthreadcore07:30:181
2995799172,12cyclictest30296-21sort07:10:161
29957991712,3cyclictest0-21swapper/108:20:141
29957991711,4cyclictest14073-21proc_pri08:30:271
2995799171,13cyclictest5299-21kthreadcore08:55:211
2995799171,12cyclictest8159-21kthreadcore07:20:201
29957991710,4cyclictest20026-21cut09:10:221
2994699176,9cyclictest15700-21tail08:35:170
2994699172,11cyclictest12750-21grep07:55:290
2994699170,11cyclictest9276-21switchtime08:25:280
2997799161,10cyclictest0-21swapper/308:40:163
2997799161,10cyclictest0-21swapper/307:15:263
2995799164,9cyclictest32302-21kthreadcore08:50:211
2995799164,9cyclictest20718-21irqcore08:40:191
29957991611,3cyclictest531-21usb-storage08:15:061
2995799161,11cyclictest15567-21fschecks_time08:35:151
2994699166,7cyclictest26741-21apt-get07:40:110
2994699162,9cyclictest0-21swapper/007:15:130
2994699161,13cyclictest22627-21kthreadcore07:35:190
2994699161,12cyclictest28404-21dpkg08:15:050
2997799151,12cyclictest0-21swapper/308:25:013
2997799141,7cyclictest0-21swapper/308:25:233
2995799141,10cyclictest29985-21latency_hist07:10:021
2995799141,10cyclictest29985-21latency_hist07:10:011
2994699141,10cyclictest30060-21cat07:10:020
2994699141,10cyclictest30060-21cat07:10:010
2997799135,6cyclictest0-21swapper/308:20:013
2997799132,6cyclictest0-21swapper/308:30:133
2997799128,2cyclictest33-21ksoftirqd/307:55:243
2997799127,3cyclictest33-21ksoftirqd/307:30:173
2997799121,9cyclictest0-21swapper/307:25:213
2997799117,2cyclictest33-21ksoftirqd/307:10:283
2997799115,4cyclictest33-21ksoftirqd/308:10:153
2997799111,7cyclictest0-21swapper/307:10:013
2997799111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional