You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 23:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Jul 16, 2026 12:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
477099571,30cyclictest10391-21ssh12:20:112
474599554,9cyclictest131rcu_preempt09:20:130
477099540,33cyclictest27090-21ssh09:30:122
4745995331,9cyclictest12-21ksoftirqd/012:30:130
4760995125,20cyclictest21-21ksoftirqd/111:35:131
4760995125,20cyclictest21-21ksoftirqd/111:35:131
477099502,16cyclictest27346irq/40-dwc2_hso10:40:142
474599485,6cyclictest131rcu_preempt10:15:110
476099471,27cyclictest1342-21ssh12:15:141
476099471,27cyclictest1342-21ssh12:15:131
477099451,11cyclictest20770-21ssh11:50:122
477099442,33cyclictest22078-21ssh11:15:132
476099402,29cyclictest16839-21ssh11:30:131
474599404,25cyclictest27386-21ssh11:00:120
4760993928,4cyclictest2742-21apt-get08:15:141
4760993915,15cyclictest21-21ksoftirqd/109:50:131
4783993825,7cyclictest35966irq/58-eth010:40:143
477099382,9cyclictest6831-21ssh11:40:302
477099382,9cyclictest6831-21ssh11:40:302
477099381,14cyclictest14948-21fschecks_time10:35:162
474599380,26cyclictest29049-21sh10:25:140
474599380,26cyclictest29049-21sh10:25:140
478399374,18cyclictest35966irq/58-eth010:20:123
478399373,21cyclictest35966irq/58-eth012:20:103
477099372,28cyclictest30667-21cat11:35:292
477099372,28cyclictest30667-21cat11:35:292
477099372,27cyclictest776-21/usr/sbin/munin10:25:302
477099372,27cyclictest776-21/usr/sbin/munin10:25:292
477099372,17cyclictest729-21usage_EthcTask12:30:292
4770993711,4cyclictest17043-21ssh11:30:162
478399363,20cyclictest35966irq/58-eth010:05:123
4783993618,12cyclictest0-21swapper/311:40:133
4783993618,12cyclictest0-21swapper/311:40:123
477099362,18cyclictest9391-21cat12:35:292
4770993612,9cyclictest1568-21cron10:10:012
4783993514,4cyclictest36066irq/59-eth011:30:123
477099359,20cyclictest4725-21cyclictest07:10:132
477099357,10cyclictest18156-21sh10:55:022
477099352,12cyclictest23091-21diskmemload11:55:122
477099351,9cyclictest21780-21latency_hist11:15:022
4770993515,12cyclictest27-21ksoftirqd/208:55:132
477099341,14cyclictest31524-21grep11:00:292
4783993320,9cyclictest0-21swapper/311:25:113
477099337,5cyclictest2889-21cat09:50:312
477099332,10cyclictest0-21swapper/212:07:302
477099331,9cyclictest14009-21cut07:50:202
477099331,8cyclictest3870-21forks09:35:162
477099322,15cyclictest10661-21unixbench_multi12:00:292
477099321,10cyclictest9330-21ntp_192.168.11410:30:252
4760993211,14cyclictest21-21ksoftirqd/107:35:121
4760993211,14cyclictest21-21ksoftirqd/107:35:121
474599321,23cyclictest12-21ksoftirqd/008:05:140
4783993121,4cyclictest35966irq/58-eth011:50:113
477099311,20cyclictest27346irq/40-dwc2_hso08:45:122
4783993023,3cyclictest0-21swapper/311:05:113
4783993023,3cyclictest0-21swapper/311:05:113
4783993020,7cyclictest35966irq/58-eth009:30:123
4783993015,10cyclictest0-21swapper/311:45:123
478399301,16cyclictest35966irq/58-eth007:35:133
478399301,16cyclictest35966irq/58-eth007:35:133
477099305,5cyclictest27784-21kthreadcore08:40:202
477099304,8cyclictest28192-21ssh10:24:292
476099303,20cyclictest2567-21ssh11:40:131
476099303,20cyclictest2567-21ssh11:40:131
476099302,22cyclictest729-21usage_EthcTask12:30:291
4760993018,6cyclictest18535-21apt-get09:05:121
476099301,4cyclictest32766-21ssh10:45:121
474599302,23cyclictest20776-21ssh11:50:130
474599302,19cyclictest9366-21ssh10:50:130
4783992922,3cyclictest0-21swapper/312:25:113
477099292,20cyclictest0-21swapper/207:55:162
477099292,20cyclictest0-21swapper/207:40:142
477099291,16cyclictest11731-21ssh11:45:122
476099293,15cyclictest16322-21ssh10:00:131
476099292,21cyclictest13609-21usage_Anybus10:50:291
4783992818,5cyclictest0-21swapper/309:25:123
4783992811,12cyclictest35966irq/58-eth008:00:123
477099283,16cyclictest27346irq/40-dwc2_hso09:20:122
477099282,11cyclictest27731-21ssh12:29:292
4770992814,6cyclictest0-21swapper/209:05:152
4770992812,13cyclictest27-21ksoftirqd/211:05:112
4770992812,13cyclictest27-21ksoftirqd/211:05:112
476099284,4cyclictest21-21ksoftirqd/109:29:341
476099282,20cyclictest19754-21ssh12:05:291
4760992810,8cyclictest4725-21cyclictest11:20:121
478399273,9cyclictest0-21swapper/312:30:293
478399273,16cyclictest35966irq/58-eth010:35:143
4783992716,5cyclictest0-21swapper/308:35:133
477099273,18cyclictest27346irq/40-dwc2_hso08:25:122
477099272,20cyclictest0-21swapper/212:13:292
477099271,5cyclictest19605-21cut10:20:012
477099271,17cyclictest27346irq/40-dwc2_hso08:30:132
4760992719,4cyclictest23718-21apt-get10:40:131
4760992715,7cyclictest22055-21apt-get11:15:121
474599272,19cyclictest9561-21ssh12:19:290
474599272,19cyclictest9561-21ssh12:19:290
474599272,19cyclictest32449-21ssh11:37:290
474599272,19cyclictest32449-21ssh11:37:290
4783992620,2cyclictest0-21swapper/311:00:113
4783992618,5cyclictest0-21swapper/310:30:133
4783992617,6cyclictest0-21swapper/311:10:123
4783992615,7cyclictest0-21swapper/309:35:143
477099263,7cyclictest0-21swapper/212:15:292
477099263,7cyclictest0-21swapper/212:15:292
477099262,21cyclictest0-21swapper/207:20:152
476099268,15cyclictest0-21swapper/110:20:121
476099267,10cyclictest21-21ksoftirqd/111:10:131
476099264,4cyclictest21-21ksoftirqd/112:10:261
476099262,18cyclictest9383-21usage_Anybus12:35:291
4760992618,4cyclictest30354-21apt-get08:10:131
4760992618,4cyclictest10961-21apt-get10:15:111
476099261,18cyclictest5676-21ssh10:30:131
4745992622,2cyclictest0-21swapper/011:40:130
4745992622,2cyclictest0-21swapper/011:40:130
474599262,19cyclictest22081-21sh11:15:130
4745992619,4cyclictest630-21Async09:36:310
478399252,18cyclictest0-21swapper/311:38:293
478399252,18cyclictest0-21swapper/311:38:293
4783992520,2cyclictest0-21swapper/311:15:113
477099258,14cyclictest1-21systemd09:10:412
477099252,19cyclictest0-21swapper/208:20:142
4770992514,9cyclictest0-21swapper/208:10:162
476099250,10cyclictest308-21ssh10:08:341
474599255,8cyclictest131rcu_preempt10:55:120
4745992519,3cyclictest0-21swapper/007:15:130
478399243,14cyclictest0-21swapper/312:15:293
478399243,14cyclictest0-21swapper/312:15:293
4783992417,5cyclictest0-21swapper/308:15:133
4783992414,7cyclictest0-21swapper/309:10:103
4783992413,8cyclictest0-21swapper/311:00:013
477099248,14cyclictest6458-21cat10:10:322
477099241,15cyclictest9794-21ls07:15:132
476099246,4cyclictest0-21swapper/109:41:131
476099243,5cyclictest21-21ksoftirqd/111:45:121
476099243,4cyclictest21-21ksoftirqd/109:55:321
476099243,11cyclictest21-21ksoftirqd/110:35:131
4760992415,4cyclictest11082-21apt-get07:50:101
4760992414,6cyclictest7787-21apt-get11:25:111
474599241,18cyclictest23736-21sh10:40:140
478399232,10cyclictest0-21swapper/312:05:143
4783992319,2cyclictest0-21swapper/308:30:113
4783992318,3cyclictest0-21swapper/312:35:123
4783992318,2cyclictest0-21swapper/310:45:123
4783992315,5cyclictest35966irq/58-eth010:25:143
4783992315,5cyclictest35966irq/58-eth010:25:143
4783992314,6cyclictest0-21swapper/310:50:283
4783992314,6cyclictest0-21swapper/310:13:233
4783992311,9cyclictest0-21swapper/311:56:203
477099238,7cyclictest29057-21grep08:05:282
477099232,16cyclictest0-21swapper/211:29:292
477099232,14cyclictest0-21swapper/207:35:062
477099232,14cyclictest0-21swapper/207:35:052
477099231,12cyclictest908-21cut10:45:182
476099237,3cyclictest0-21swapper/112:20:341
476099237,3cyclictest0-21swapper/111:51:031
476099237,3cyclictest0-21swapper/109:16:151
476099237,3cyclictest0-21swapper/109:16:151
476099236,5cyclictest21-21ksoftirqd/108:25:011
476099236,4cyclictest0-21swapper/109:34:221
476099236,3cyclictest0-21swapper/109:14:281
476099233,4cyclictest21-21ksoftirqd/109:46:241
476099232,6cyclictest31163-21ssh10:25:201
476099232,6cyclictest31163-21ssh10:25:201
476099231,6cyclictest21-21ksoftirqd/112:25:101
4760992316,3cyclictest27376-21apt-get11:00:121
4760992315,4cyclictest14569-21apt-get07:20:141
4760992314,6cyclictest10608-21sshd12:00:291
474599233,8cyclictest131rcu_preempt08:20:140
4745992318,3cyclictest0-21swapper/010:00:130
4745992313,7cyclictest0-21swapper/010:50:020
474599231,18cyclictest26901-21cut08:40:160
478399227,12cyclictest0-21swapper/312:10:133
4783992218,2cyclictest0-21swapper/308:50:113
4783992218,2cyclictest0-21swapper/308:50:113
4783992217,2cyclictest0-21swapper/309:55:123
4783992216,2cyclictest0-21swapper/307:55:143
4783992213,6cyclictest0-21swapper/311:23:353
4783992213,6cyclictest0-21swapper/309:40:333
4783992213,6cyclictest0-21swapper/308:05:243
4783992212,7cyclictest0-21swapper/309:45:113
4783992210,9cyclictest0-21swapper/312:04:203
477099226,3cyclictest167-21systemd-network08:38:532
4770992212,7cyclictest0-21swapper/208:50:162
4770992212,7cyclictest0-21swapper/208:50:162
477099221,11cyclictest20467-21sed10:00:302
477099220,15cyclictest23091-21diskmemload09:47:112
476099226,3cyclictest0-21swapper/109:22:011
476099225,4cyclictest0-21swapper/107:30:121
476099224,5cyclictest21-21ksoftirqd/110:55:121
476099224,4cyclictest21-21ksoftirqd/111:06:111
476099224,4cyclictest21-21ksoftirqd/111:06:101
476099223,6cyclictest21-21ksoftirqd/108:00:271
476099223,4cyclictest21-21ksoftirqd/110:11:451
474599221,11cyclictest12506-21cut12:20:180
4783992117,2cyclictest0-21swapper/308:40:123
4783992115,4cyclictest0-21swapper/310:00:133
4783992114,3cyclictest0-21swapper/310:15:123
4783992113,6cyclictest0-21swapper/308:10:243
4783992112,7cyclictest0-21swapper/309:20:013
4783992112,7cyclictest0-21swapper/309:20:013
4783992112,6cyclictest0-21swapper/309:52:083
4783992112,6cyclictest0-21swapper/308:25:303
4783992112,6cyclictest0-21swapper/307:50:193
4783992111,7cyclictest0-21swapper/308:20:293
4783992111,6cyclictest0-21swapper/307:15:183
477099217,3cyclictest0-21swapper/208:15:222
4770992113,5cyclictest0-21swapper/210:55:272
4770992112,7cyclictest0-21swapper/209:00:232
4770992112,6cyclictest0-21swapper/208:00:272
4770992110,9cyclictest4918-21ssh11:22:052
4770992110,8cyclictest2086-21kthreadcore09:15:222
4770992110,8cyclictest2086-21kthreadcore09:15:222
476099216,3cyclictest0-21swapper/111:55:051
476099212,4cyclictest21-21ksoftirqd/108:40:181
4760992115,3cyclictest13642-21apt-get09:00:121
4760992115,3cyclictest12349-21apt-get08:25:121
4760992114,4cyclictest17159-21apt-get08:30:121
474599218,7cyclictest131rcu_preempt08:10:120
474599218,10cyclictest15264-21cron12:05:010
474599212,15cyclictest4924-21apt-get07:10:120
4745992112,3cyclictest23091-21diskmemload12:27:010
474599211,16cyclictest1538-21ls07:40:140
478399209,8cyclictest0-21swapper/309:20:153
4783992016,2cyclictest0-21swapper/309:05:113
4783992015,3cyclictest0-21swapper/308:55:133
4783992013,5cyclictest0-21swapper/307:30:113
4783992011,6cyclictest0-21swapper/307:25:183
4783992011,6cyclictest0-21swapper/307:10:303
4770992013,5cyclictest0-21swapper/207:25:152
4770992013,4cyclictest0-21swapper/209:40:302
4770992011,6cyclictest0-21swapper/209:55:272
477099201,11cyclictest19980-21kthreadcore09:25:202
477099201,10cyclictest8293-21kthreadcore07:45:202
476099208,7cyclictest0-21swapper/108:35:151
476099203,12cyclictest0-21swapper/108:55:141
476099202,9cyclictest21-21ksoftirqd/109:35:511
476099201,5cyclictest0-21swapper/107:45:171
4760992012,5cyclictest0-21swapper/107:25:271
474599202,15cyclictest22020-21ps09:05:260
474599202,14cyclictest15680-21ssh12:05:140
474599201,16cyclictest5247-21ssh12:35:130
4745992011,6cyclictest25608-21ssh11:35:010
4745992011,6cyclictest20492-21cron08:00:010
4783991915,2cyclictest0-21swapper/309:00:123
4783991915,2cyclictest0-21swapper/307:20:183
4783991914,3cyclictest0-21swapper/307:45:153
4783991914,3cyclictest0-21swapper/307:40:183
4783991910,6cyclictest0-21swapper/308:45:173
477099191,10cyclictest25816-21cut07:30:192
476099193,3cyclictest21-21ksoftirqd/107:55:271
4760991912,3cyclictest3941-21apt-get08:50:121
4760991912,3cyclictest3941-21apt-get08:50:121
4760991912,3cyclictest25575-21apt-get08:05:131
474599199,7cyclictest22049-21df08:35:140
474599199,7cyclictest15660-21sshd07:50:290
474599198,8cyclictest27854-21cat09:10:290
474599197,3cyclictest131rcu_preempt10:10:320
474599194,6cyclictest12-21ksoftirqd/011:45:240
474599193,11cyclictest12-21ksoftirqd/011:05:120
474599193,11cyclictest12-21ksoftirqd/011:05:120
474599192,14cyclictest24753-21ssh12:10:160
474599192,13cyclictest25346-21ntpq09:45:250
474599191,9cyclictest0-21swapper/009:35:060
474599191,15cyclictest4123-21cut08:50:140
474599191,15cyclictest4123-21cut08:50:140
4745991911,5cyclictest14524-21kthreadcore09:40:220
4745991910,6cyclictest11405-21usage_Anybus09:55:310
4745991910,6cyclictest0-21swapper/009:15:160
4745991910,6cyclictest0-21swapper/009:15:150
4745991910,5cyclictest23306-21sh10:40:010
476099186,9cyclictest13550-21grep07:15:281
476099186,3cyclictest0-21swapper/107:10:121
476099182,4cyclictest21-21ksoftirqd/108:45:181
474599189,7cyclictest14794-21wc07:20:180
474599188,7cyclictest18544-21cut09:25:170
474599187,8cyclictest585-21kthreadcore11:55:210
474599187,3cyclictest131rcu_preempt10:20:200
474599183,4cyclictest11755-21switchtime11:25:290
474599183,11cyclictest17455-21unixbench_multi09:00:290
474599182,12cyclictest588-21kthreadcore11:20:200
474599181,8cyclictest25316-21/usr/sbin/munin10:05:150
474599181,7cyclictest135-21systemd-journal09:55:010
4745991812,4cyclictest0-21swapper/007:35:150
4745991812,4cyclictest0-21swapper/007:35:150
474599181,12cyclictest16754-21ntp_states11:10:260
474599181,11cyclictest27296-21uniq09:30:160
477099170,10cyclictest4674-21latency_hist07:10:022
476099175,9cyclictest4859-21ntp_192.168.11407:40:241
474599179,6cyclictest8521-21cron08:55:040
474599173,11cyclictest19501-21kthreadcore08:30:210
474599172,13cyclictest5649-21apt-get10:30:120
474599172,10cyclictest0-21swapper/008:15:140
474599171,12cyclictest12442-21diskstats08:25:140
474599171,12cyclictest0-21swapper/007:45:130
4783991611,3cyclictest0-21swapper/307:10:023
474599161,12cyclictest25999-21kthreadcore07:30:190
474599161,12cyclictest22762-21ntp_192.168.11407:25:240
474599152,9cyclictest0-21swapper/008:45:140
476099141,10cyclictest4763-21date07:10:031
474599141,10cyclictest4757-21latency_hist07:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional