You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-22 - 13:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Jul 22, 2026 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2369099604,42cyclictest27763-21ssh22:50:120
23712995531,15cyclictest35966irq/58-eth022:55:133
2369199541,43cyclictest29711-21ssh23:45:141
23690995136,10cyclictest8103-21apt-get23:15:140
23690994938,7cyclictest11690-21apt-get23:35:120
2369199471,34cyclictest4931-21ssh00:25:141
2369199466,4cyclictest131rcu_preempt21:45:131
2369199464,4cyclictest131rcu_preempt20:25:141
23690994529,12cyclictest4921-21ssh00:25:140
2370399442,33cyclictest29705-21ssh23:45:132
23690994437,3cyclictest28742-21apt-get19:15:150
23690994435,4cyclictest7093-21apt-get20:00:120
23690994435,4cyclictest4794-21apt-get21:05:160
23690994418,11cyclictest29410-21ssh21:38:070
23690994317,17cyclictest11127-21ssh22:57:510
23690994316,11cyclictest26781-21ssh23:42:000
23690994227,9cyclictest0-21swapper/020:50:120
2369099422,31cyclictest17165-21ssh23:20:130
23690994216,11cyclictest17954-21ssh21:14:330
2370399411,23cyclictest11701-21ssh23:35:122
2369099412,31cyclictest27811-21ssh21:20:150
23690994118,16cyclictest0-21swapper/000:30:130
23690994116,15cyclictest30320-21ssh23:09:110
23690994116,10cyclictest5174-21ssh22:19:270
23690994115,10cyclictest26341-21ssh22:11:430
2371299403,23cyclictest35966irq/58-eth022:25:143
2369199404,25cyclictest4332-21ssh22:55:151
23690994020,12cyclictest12-21ksoftirqd/022:40:130
23690994016,10cyclictest23686-21ssh22:29:560
23690994013,9cyclictest29617-21ssh00:00:470
23690994013,11cyclictest11567-21rm22:21:130
2371299392,24cyclictest35966irq/58-eth023:45:153
23712993916,14cyclictest35966irq/58-eth023:35:123
2370399393,32cyclictest816-21ssh22:35:142
2370399392,29cyclictest19226-21ssh22:45:182
23690993925,9cyclictest29915-21apt-get19:50:110
23690993916,7cyclictest22276-21ssh21:51:020
23690993915,8cyclictest32659-21ssh22:34:490
23690993912,10cyclictest20057-21ssh22:09:580
23690993816,8cyclictest9343-21diskmemload23:34:180
23690993815,7cyclictest28519-21ntp_192.168.11419:45:250
23690993814,9cyclictest9129-21ssh22:02:320
23690993814,8cyclictest9343-21diskmemload23:56:510
23690993814,8cyclictest6719-21ssh00:06:220
23690993813,9cyclictest19992-21ssh23:02:390
23690993714,9cyclictest32728-21sh23:27:350
23690993714,10cyclictest7829-21rm22:38:100
23690993713,8cyclictest13758-21cut19:30:210
23690993713,17cyclictest30050-21if_eth023:45:180
23690993712,8cyclictest4254-21ssh21:40:560
23690993712,8cyclictest0-21swapper/021:15:080
23690993710,13cyclictest11899-21grep21:45:280
23691993610,16cyclictest21-21ksoftirqd/122:45:141
23690993624,9cyclictest1668-21ssh00:21:380
23690993617,6cyclictest18540-21ssh21:30:550
23690993615,16cyclictest0-21swapper/020:55:130
23690993614,8cyclictest26653-21ssh00:18:330
23690993614,8cyclictest16209-21ssh00:11:560
23690993613,8cyclictest0-21swapper/022:46:540
23690993611,8cyclictest0-21swapper/000:35:370
23690993610,8cyclictest0-21swapper/023:52:200
23691993527,5cyclictest20666-21ssh23:40:131
23690993526,5cyclictest12-21ksoftirqd/020:40:150
23690993515,4cyclictest21357-21ntp_192.168.11420:45:240
23690993514,9cyclictest31561-21sh21:56:140
23690993514,7cyclictest9343-21diskmemload21:25:210
2371299343,15cyclictest35966irq/58-eth000:15:133
2371299341,22cyclictest35966irq/58-eth000:25:153
2370399343,9cyclictest0-21swapper/222:42:142
2369199345,6cyclictest131rcu_preempt21:50:141
23691993419,6cyclictest0-21swapper/121:35:151
23691993416,8cyclictest21-21ksoftirqd/100:05:131
23690993421,8cyclictest0-21swapper/021:00:130
23690993420,10cyclictest6127-21fw_packets19:25:170
23690993419,11cyclictest2959-21cut19:55:200
2371299338,16cyclictest35966irq/58-eth021:45:133
2369199333,27cyclictest15358-21ssh23:55:151
23690993325,6cyclictest21548-21/usr/sbin/munin20:15:160
23690993317,4cyclictest6778-21ssh23:13:550
2371299323,17cyclictest35966irq/58-eth000:05:133
2370399322,19cyclictest27346irq/40-dwc2_hso22:30:152
2369199323,26cyclictest20523-21ssh22:10:141
23690993219,9cyclictest4043-21cut19:20:210
23690993212,7cyclictest0-21swapper/020:35:250
23690993211,8cyclictest15467-21/usr/sbin/munin19:35:290
2371299312,20cyclictest35966irq/58-eth019:55:143
23712993119,7cyclictest0-21swapper/319:50:113
2369199319,8cyclictest131rcu_preempt23:50:141
2369199311,25cyclictest18731-21ssh21:15:121
2369099313,19cyclictest12-21ksoftirqd/019:10:130
23690993117,11cyclictest19208-21cut20:10:230
2369199307,19cyclictest2664-21ssh22:00:131
2369199307,13cyclictest630-21Async09:36:311
2369199303,22cyclictest26098-21ssh21:55:151
2369199302,21cyclictest31519-21ssh21:40:141
23691993012,8cyclictest21-21ksoftirqd/123:35:131
23690993019,8cyclictest32350-21kthreadcore20:25:230
23690993019,8cyclictest26467-21/usr/sbin/munin20:20:150
23690993019,8cyclictest26467-21/usr/sbin/munin20:20:150
2370399294,18cyclictest27346irq/40-dwc2_hso20:00:132
2370399292,14cyclictest8007-21sh21:45:132
2369199291,25cyclictest31451-21ssh23:10:141
2369199291,25cyclictest28320-21ssh00:20:131
23690992924,2cyclictest20288-21apt-get19:40:120
23690992921,5cyclictest6950-21ntp_192.168.11420:30:240
23690992919,7cyclictest11972-21df20:05:140
2370399282,21cyclictest0-21swapper/220:55:162
2370399282,20cyclictest0-21swapper/220:15:152
2370399282,20cyclictest0-21swapper/219:45:162
2370399282,11cyclictest0-21swapper/222:51:182
23703992815,10cyclictest0-21swapper/220:20:182
23703992815,10cyclictest0-21swapper/220:20:182
23703992814,10cyclictest0-21swapper/223:15:282
2370399281,13cyclictest19242-21ssh00:15:142
2370399280,12cyclictest9343-21diskmemload23:25:122
2369199285,3cyclictest9343-21diskmemload00:00:141
2369199283,21cyclictest14035-21ssh00:30:131
2369199282,22cyclictest26094-21apt-get23:25:121
2369199280,19cyclictest0-21swapper/120:45:141
2370399273,17cyclictest27346irq/40-dwc2_hso20:50:122
2370399272,9cyclictest0-21swapper/200:25:142
2370399272,21cyclictest0-21swapper/222:56:192
2370399272,20cyclictest32587-21ssh00:04:192
2370399272,10cyclictest0-21swapper/221:05:162
2369199275,13cyclictest2700-21ssh23:30:131
2369199272,9cyclictest131rcu_preempt23:05:121
2369199272,19cyclictest14450-21ssh23:17:181
23690992716,9cyclictest23689-21cyclictest19:10:010
2370399262,16cyclictest27346irq/40-dwc2_hso19:55:162
2370399262,10cyclictest0-21swapper/219:25:152
2370399261,18cyclictest25947-21ssh21:55:042
2370399253,17cyclictest27346irq/40-dwc2_hso22:05:152
2370399253,17cyclictest0-21swapper/222:27:142
2370399252,15cyclictest27346irq/40-dwc2_hso19:40:122
23703992517,5cyclictest23994-21ssh00:00:012
2369199252,18cyclictest9343-21diskmemload22:20:151
2369199252,14cyclictest15130-21ssh22:25:141
2370399249,12cyclictest0-21swapper/223:25:002
23703992412,9cyclictest11004-21ssh22:20:322
2369199243,18cyclictest13387-21ssh23:00:151
2369199242,19cyclictest23085-21ssh00:35:141
23712992314,6cyclictest35966irq/58-eth021:25:153
2370399239,7cyclictest31231-21df20:25:162
2370399239,11cyclictest25992-21awk23:05:242
2370399232,8cyclictest13397-21ssh23:00:152
2370399232,6cyclictest0-21swapper/222:19:182
2370399232,16cyclictest0-21swapper/223:13:182
2370399232,16cyclictest0-21swapper/200:11:192
2370399232,16cyclictest0-21swapper/200:07:282
2370399232,14cyclictest0-21swapper/219:20:122
23703992312,9cyclictest13650-21ssh23:53:272
2370399231,15cyclictest14308-21ls19:30:272
2369199232,17cyclictest32403-21apt-get21:00:141
2369199231,18cyclictest28733-21find19:15:141
2371299223,13cyclictest0-21swapper/323:07:213
2371299222,16cyclictest35966irq/58-eth023:00:143
2371299221,11cyclictest35966irq/58-eth023:50:143
2370399223,11cyclictest15168-21wc20:05:222
2370399222,6cyclictest0-21swapper/223:43:182
2370399222,5cyclictest0-21swapper/220:40:152
2370399222,15cyclictest0-21swapper/221:31:172
2370399222,12cyclictest13555-21apt-get00:30:062
23703992211,9cyclictest27-21ksoftirqd/221:20:142
23703992211,9cyclictest22186-21ssh21:35:022
23703992211,8cyclictest0-21swapper/220:30:282
2370399221,13cyclictest26767-21ntp_states00:35:262
2370399221,12cyclictest24676-21grep22:10:302
2370399221,11cyclictest6401-21sh23:30:262
2369199222,18cyclictest20506-21ssh21:33:181
2369199221,16cyclictest29788-21ls19:50:021
23691992210,3cyclictest131rcu_preempt23:20:201
2371299219,6cyclictest35966irq/58-eth000:10:133
2371299212,14cyclictest0-21swapper/300:35:153
2371299211,15cyclictest35966irq/58-eth023:10:153
2370399217,6cyclictest3436-21kthreadcore22:00:212
2370399211,17cyclictest18907-21kthreadcore20:45:212
2370399211,16cyclictest0-21swapper/221:00:142
2370399211,14cyclictest27346irq/40-dwc2_hso19:10:132
23703992110,5cyclictest9394-21ssh21:25:452
2369199218,10cyclictest15253-21cron19:35:021
2369199213,15cyclictest3871-21kthreadcore19:55:211
2369199213,12cyclictest0-21swapper/119:35:121
2369199212,16cyclictest9343-21diskmemload00:10:181
23691992114,4cyclictest31477-21kthreadcore22:15:221
23691992114,4cyclictest0-21swapper/122:05:161
2369199211,14cyclictest27827-21ssh21:20:171
2369199211,12cyclictest4869-21ls21:05:161
2369199211,11cyclictest26512-21cut22:30:231
2371299208,6cyclictest0-21swapper/322:19:563
2371299205,9cyclictest35966irq/58-eth023:40:133
2371299205,9cyclictest0-21swapper/300:00:143
2371299204,12cyclictest0-21swapper/319:15:143
2371299203,14cyclictest0-21swapper/323:29:183
2371299203,12cyclictest0-21swapper/321:35:143
2371299203,11cyclictest0-21swapper/322:54:183
2371299202,11cyclictest0-21swapper/319:35:133
23712992011,7cyclictest0-21swapper/321:53:053
23712992011,6cyclictest0-21swapper/321:30:183
23712992010,4cyclictest0-21swapper/322:00:133
2370399203,9cyclictest32171-21irqrtprio21:40:192
2370399202,8cyclictest32005-21kthreadcore19:50:182
2370399201,9cyclictest30690-21kthreadcore00:20:212
2370399201,8cyclictest9773-21rm21:10:152
23703992015,3cyclictest20118-21rm21:50:222
2370399201,16cyclictest936-21latency_hist19:20:022
23703992010,7cyclictest9422-21sshd20:35:182
23691992012,5cyclictest19578-21if_eth100:15:191
2369199201,17cyclictest19910-21ls20:10:231
23691992010,7cyclictest18347-21cron21:15:021
2371299199,7cyclictest0-21swapper/322:40:153
2371299193,11cyclictest0-21swapper/323:21:193
2371299193,11cyclictest0-21swapper/323:17:223
23712991912,2cyclictest0-21swapper/320:00:133
23712991911,5cyclictest0-21swapper/322:35:193
2371299191,12cyclictest0-21swapper/320:40:253
23712991910,6cyclictest0-21swapper/322:46:303
2370399192,8cyclictest18747-21kthreadcore20:10:212
2370399191,16cyclictest19167-21irqcore21:15:182
2370399191,15cyclictest15668-21fschecks_time19:35:162
2370399191,10cyclictest23855-21latency_hist19:10:022
2369199196,10cyclictest614-21dbus-daemon20:06:441
2369199194,11cyclictest28457-21kthreadcore20:20:211
2369199194,11cyclictest28457-21kthreadcore20:20:211
2369199192,13cyclictest13299-21fschecks_count20:40:191
2369199191,7cyclictest28515-21munin-run19:15:021
23691991910,6cyclictest3130-21kworker/1:021:25:161
2371299189,7cyclictest0-21swapper/300:31:013
2371299189,7cyclictest0-21swapper/300:20:453
2371299189,6cyclictest0-21swapper/323:30:483
2371299189,6cyclictest0-21swapper/321:15:133
2371299182,11cyclictest0-21swapper/322:32:153
2371299182,10cyclictest0-21swapper/323:55:143
2371299182,10cyclictest0-21swapper/320:25:153
2371299181,14cyclictest0-21swapper/319:50:023
2371299181,13cyclictest0-21swapper/319:20:213
2371299181,12cyclictest0-21swapper/322:05:213
23712991810,6cyclictest0-21swapper/322:20:133
23712991810,6cyclictest0-21swapper/321:55:313
2369199185,10cyclictest1755-21kernelversion19:20:191
2369199183,11cyclictest18438-21munin-plugin-st22:45:021
2369199182,12cyclictest4574-21kthreadcore20:30:201
2369199181,8cyclictest27692-21fschecks_count20:55:171
2369199181,14cyclictest7078-21find20:00:111
2369199181,14cyclictest29902-21cut22:50:211
2369199181,14cyclictest26034-21cron20:20:011
2369199181,13cyclictest22757-21apt-get20:50:131
2371299179,6cyclictest0-21swapper/322:10:373
2371299179,6cyclictest0-21swapper/321:24:403
2371299172,9cyclictest0-21swapper/321:05:163
2371299172,10cyclictest0-21swapper/320:10:163
23712991710,3cyclictest35966irq/58-eth021:10:143
2369199175,7cyclictest0-21swapper/119:50:111
2369199174,10cyclictest8486-21kthreadcore19:25:221
2369199172,13cyclictest25018-21ls19:45:021
2369199171,11cyclictest8745-21irqcore20:35:161
2371299169,5cyclictest35966irq/58-eth021:40:323
2371299166,7cyclictest0-21swapper/319:40:183
2369199161,11cyclictest23777-21grep19:10:011
2371299151,11cyclictest0-21swapper/320:45:283
2371299149,3cyclictest0-21swapper/320:59:553
2371299149,3cyclictest0-21swapper/320:50:313
2371299143,8cyclictest35966irq/58-eth020:15:153
2371299141,7cyclictest0-21swapper/320:20:153
2371299141,7cyclictest0-21swapper/320:20:143
2371299141,10cyclictest0-21swapper/319:25:233
2371299138,3cyclictest0-21swapper/320:30:233
2371299131,6cyclictest0-21swapper/320:35:263
2371299123,7cyclictest0-21swapper/319:10:163
2371299121,8cyclictest0-21swapper/321:00:203
2371299121,7cyclictest0-21swapper/320:10:013
2371299121,7cyclictest0-21swapper/319:10:013
2371299111,7cyclictest0-21swapper/319:30:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional