You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 22:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Mon Jun 01, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12833217715,117sleep112897-21grep06:55:461
11919212512,71sleep211920-21/usr/sbin/munin06:55:102
134289910443,55cyclictest28337-21kworker/u18:1+events_unbound10:49:582
13428991022,47cyclictest16958-21ssh11:35:082
134289910036,12cyclictest0-21swapper/210:35:002
1342899993,37cyclictest24378-21taskset09:06:112
1342899983,41cyclictest32210-21fschecks_count07:55:232
1342899983,36cyclictest5878-21latency_hist08:15:012
1342899977,40cyclictest13625-21idleruntime-cro11:29:582
1342899974,33cyclictest1915-21/usr/sbin/munin11:10:252
1342899974,33cyclictest1915-21/usr/sbin/munin11:10:252
1342899973,33cyclictest20459-21latency_hist09:00:022
13428999674,16cyclictest22698-21sh11:45:002
13428999674,16cyclictest22698-21sh11:45:002
13428999529,11cyclictest0-21swapper/209:30:142
13428999524,25cyclictest20895-21cat07:20:242
1342899952,35cyclictest27384-21latency_hist10:05:012
1342899952,35cyclictest27384-21latency_hist10:05:012
1342899952,33cyclictest30623-21fschecks_time07:50:192
13428999452,35cyclictest3128-21systemd-run09:25:002
1342899943,37cyclictest25820-21fschecks_time07:35:212
1342899943,36cyclictest13298-21/usr/sbin/munin10:35:322
1342899943,35cyclictest8624-21latency_hist12:15:002
13428999428,12cyclictest0-21swapper/208:25:162
1342899933,34cyclictest19116-21grep07:15:202
1342899933,34cyclictest14774-21/usr/sbin/munin12:25:322
1342899933,32cyclictest365-21ntpq12:00:262
13428999330,11cyclictest0-21swapper/210:10:222
1342899932,34cyclictest27418-21tune2fs07:40:212
13428999251,34cyclictest15172-21munin-node11:05:212
1342899924,32cyclictest12578-21cstates08:35:182
13428999235,10cyclictest0-21swapper/207:50:012
13428999235,10cyclictest0-21swapper/207:50:012
1342899923,39cyclictest30329-21grep09:15:262
1342899923,39cyclictest15409-21munin-node07:05:062
1342899923,39cyclictest15409-21munin-node07:05:062
1342899923,36cyclictest352-21runrttasks10:20:242
1342899923,36cyclictest23634-21latency_hist07:30:012
1342899923,35cyclictest7282-21/usr/sbin/munin10:25:212
1342899923,35cyclictest7282-21/usr/sbin/munin10:25:212
1342899923,31cyclictest1608-21fschecks_time10:15:212
1342899923,31cyclictest1608-21fschecks_time10:15:212
13428999171,13cyclictest13916-21sshd09:40:592
1342899914,34cyclictest8420-21timerandwakeup08:20:342
1342899914,34cyclictest8420-21timerandwakeup08:20:342
13428999137,10cyclictest13405-21cyclictest10:05:242
1342899913,37cyclictest1993-21grep08:00:332
13428999126,14cyclictest0-21swapper/208:49:592
1342899912,35cyclictest28209-21latency_hist11:00:012
1342899912,34cyclictest17363-21cstates07:10:152
13428999110,55cyclictest23272-21ssh11:45:232
13415999145,33cyclictest181rcu_preempt10:31:400
1342899904,32cyclictest4902-21/usr/sbin/munin11:15:152
1342899903,35cyclictest25293-21latency_hist07:35:012
1342899903,33cyclictest13573-21/usr/sbin/munin07:00:092
1342899894,34cyclictest352-21runrttasks08:06:372
1342899893,38cyclictest10800-21latency_hist11:25:012
1342899893,35cyclictest31981-21latency_hist12:00:002
1342899893,35cyclictest31981-21latency_hist12:00:002
1342899893,35cyclictest18218-21unixbench_multi08:50:352
1342899893,33cyclictest6081-21latency_hist09:30:002
1342899893,31cyclictest16568-21fschecks_count10:40:192
1342899893,31cyclictest16568-21fschecks_count10:40:192
13428998927,15cyclictest0-21swapper/209:45:372
1342899892,34cyclictest23050-21processes10:50:322
1342899892,33cyclictest14073-21/usr/sbin/munin08:40:162
13419998914,59cyclictest23187-21apt-get11:45:161
13415998913,64cyclictest12672-21apt-get09:40:220
13415998911,68cyclictest23981-21sshd07:30:250
1342899884,38cyclictest12554-21apt-key09:40:032
1342899884,32cyclictest22140-21rm09:55:232
1342899883,32cyclictest2881-21cstates12:05:142
1342899882,37cyclictest12300-21latency_hist08:35:002
1341599883,52cyclictest12014-21grep12:20:190
1342899876,33cyclictest29533-21cron09:14:572
1342899876,33cyclictest29533-21cron09:14:572
1342899874,34cyclictest28325-21/usr/sbin/munin11:00:352
1342899873,34cyclictest6257-21fschecks_time08:15:202
1342899873,34cyclictest6257-21fschecks_time08:15:202
1342899873,33cyclictest16303-21sh11:34:032
1342899873,33cyclictest16303-21sh11:34:032
13419998762,18cyclictest13405-21cyclictest11:30:321
13419998762,18cyclictest13405-21cyclictest11:30:321
13415998766,14cyclictest4232-21latency_hist08:10:010
1342899864,32cyclictest9880-21grep12:15:362
13428998525,12cyclictest0-21swapper/209:50:202
13428998525,12cyclictest0-21swapper/209:50:202
13415998567,14cyclictest22001-21kworker/u18:3+events_unbound07:49:590
13415998567,14cyclictest22001-21kworker/u18:3+events_unbound07:49:590
13415998565,13cyclictest25742-21sh10:55:230
13419998437,36cyclictest32200-21/usr/sbin/munin12:00:201
13415998462,8cyclictest181rcu_preempt11:53:250
1342899833,31cyclictest12018-21fschecks_count12:20:192
1341599833,42cyclictest15-21ksoftirqd/008:30:110
13415998260,8cyclictest181rcu_preempt11:39:150
13419998162,13cyclictest5853-21timerwakeupswit11:15:341
1341999814,55cyclictest23846-21apt-get09:05:481
1342899804,30cyclictest29146-21latency_hist11:55:012
13419998058,13cyclictest20294-21diskmemload09:30:261
13419998022,17cyclictest0-21swapper/109:30:021
13419997952,21cyclictest20294-21diskmemload10:50:001
13419997952,17cyclictest14114-21ntp_offset07:00:271
13415997961,13cyclictest20294-21diskmemload11:40:200
13415997961,13cyclictest20294-21diskmemload11:40:200
13415997959,15cyclictest9359-21latency_hist09:35:020
13415997955,7cyclictest181rcu_preempt09:47:300
13415997954,11cyclictest181rcu_preempt07:19:220
13415997937,16cyclictest181rcu_preempt12:04:070
13428997854,18cyclictest20798-21sh09:00:172
13419997863,10cyclictest6987-21modprobe10:24:571
13415997856,8cyclictest181rcu_preempt09:04:080
13415997856,8cyclictest181rcu_preempt08:04:590
13415997839,12cyclictest181rcu_preempt10:25:100
13415997839,12cyclictest181rcu_preempt10:25:100
13415997834,17cyclictest181rcu_preempt08:20:170
13415997834,17cyclictest181rcu_preempt08:20:170
13415997745,8cyclictest181rcu_preempt09:50:590
13415997745,8cyclictest181rcu_preempt09:50:590
13419997653,12cyclictest0-21swapper/111:10:331
13419997653,12cyclictest0-21swapper/111:10:331
13419997648,20cyclictest12428-21apt-key09:40:031
1341599766,54cyclictest13417-21latency_hist07:00:020
13415997648,7cyclictest181rcu_preempt08:40:230
13415997634,16cyclictest181rcu_preempt10:07:320
1341599763,35cyclictest4588-21cron11:14:570
1341599763,35cyclictest4588-21cron11:14:570
1341599762,37cyclictest5997-21ssh10:22:150
13419997556,13cyclictest16556-21threads09:45:361
13419997553,16cyclictest960-21ntp_kernel_err09:20:261
13419997552,15cyclictest23477-21apt-config09:05:011
13415997557,11cyclictest298-21in:imuxsock07:54:590
13415997554,13cyclictest13405-21cyclictest09:13:330
13415997554,13cyclictest13405-21cyclictest09:13:330
13415997553,13cyclictest31021-21ssh11:57:310
13415997553,13cyclictest31021-21ssh11:57:310
13415997546,14cyclictest181rcu_preempt08:28:480
13415997534,7cyclictest181rcu_preempt12:27:340
13415997530,13cyclictest181rcu_preempt10:35:180
13419997445,22cyclictest27964-21kworker/u18:2+nfsiod10:59:511
13419997429,10cyclictest0-21swapper/111:40:341
13419997429,10cyclictest0-21swapper/111:40:341
13415997444,8cyclictest181rcu_preempt12:19:210
13415997441,9cyclictest0-21swapper/007:05:060
13415997441,9cyclictest0-21swapper/007:05:060
13415997438,18cyclictest181rcu_preempt12:07:580
13415997434,13cyclictest181rcu_preempt10:53:450
13415997425,27cyclictest181rcu_preempt11:24:310
1341999737,48cyclictest15364-21ssh09:44:581
1341999731,38cyclictest27216-21grep11:50:351
13415997358,10cyclictest13405-21cyclictest07:20:200
13415997354,13cyclictest1009-21idleruntime-cro10:14:580
13415997332,19cyclictest181rcu_preempt07:29:590
13415997332,12cyclictest181rcu_preempt09:16:570
13415997331,15cyclictest29108-21run-parts11:00:320
1341999728,47cyclictest29551-21sh09:14:571
1341999728,47cyclictest29551-21sh09:14:571
1341999726,27cyclictest352-21runrttasks10:51:091
13419997258,10cyclictest13405-21cyclictest11:55:181
13419997258,10cyclictest13405-21cyclictest11:55:181
13419997255,10cyclictest12367-21ntp_kernel_pll_12:20:261
1341999723,49cyclictest30743-21/usr/sbin/munin10:10:311
13415997251,8cyclictest181rcu_preempt09:36:060
13415997251,15cyclictest23728-21cut11:46:000
1341599723,43cyclictest352-21runrttasks10:49:320
1341599722,41cyclictest18152-21runrttasks10:42:360
1341599722,41cyclictest18152-21runrttasks10:42:360
13419997156,8cyclictest0-21swapper/109:55:191
13419997153,12cyclictest277-21dbus-daemon07:30:001
13419997152,13cyclictest27645-21ntp_kernel_err07:40:261
13419997149,16cyclictest17562-21fschecks_time08:50:201
13419997149,12cyclictest0-21swapper/110:25:331
13419997149,12cyclictest0-21swapper/110:25:331
1341999714,45cyclictest17465-21cut07:10:191
1341999712,22cyclictest352-21runrttasks07:19:361
13419997119,24cyclictest20294-21diskmemload11:09:261
13419997114,33cyclictest11511-21ntpq11:25:251
13415997149,8cyclictest181rcu_preempt09:25:340
13415997133,17cyclictest181rcu_preempt07:35:170
13436997047,16cyclictest451rcuc/307:57:033
13419997051,14cyclictest7591-21kworker/u19:0+events_unbound10:09:581
13419997047,17cyclictest20294-21diskmemload12:25:231
13419997046,16cyclictest13405-21cyclictest08:05:221
1341999703,50cyclictest4991-21wc08:10:301
1341999702,38cyclictest0-21swapper/111:36:541
13419997014,41cyclictest31037-21apt11:04:581
1341599709,37cyclictest15-21ksoftirqd/011:25:170
1341599709,24cyclictest15-21ksoftirqd/011:31:400
1341599709,24cyclictest15-21ksoftirqd/011:31:400
13415997057,9cyclictest13405-21cyclictest07:41:210
1341599705,31cyclictest15-21ksoftirqd/008:10:210
13415997052,12cyclictest23824-21ssh09:05:080
13415997028,7cyclictest181rcu_preempt08:59:240
13415997028,13cyclictest181rcu_preempt10:19:520
13415997028,13cyclictest181rcu_preempt10:19:520
1341999698,30cyclictest0-21swapper/108:03:051
13419996949,15cyclictest23879-21kworker/u18:0+nfsiod10:00:191
13419996949,15cyclictest23879-21kworker/u18:0+nfsiod10:00:191
13419996949,14cyclictest20294-21diskmemload11:20:261
1341999693,43cyclictest30178-21fschecks_time09:15:221
13419996924,32cyclictest17086-21cat07:10:001
13419996924,32cyclictest17086-21cat07:10:001
13419996918,22cyclictest20194-21ssh09:51:401
13419996918,22cyclictest20194-21ssh09:51:401
13415996952,11cyclictest20294-21diskmemload09:24:590
13415996936,15cyclictest181rcu_preempt08:35:230
1341599692,34cyclictest32187-21switchtime11:05:320
13415996917,16cyclictest15-21ksoftirqd/008:15:340
13415996917,16cyclictest15-21ksoftirqd/008:15:340
1341999689,25cyclictest21020-21ntp_states07:20:291
13419996846,16cyclictest5475-21cut12:10:001
1341999684,42cyclictest13598-21uniq10:35:201
1341999683,44cyclictest486-21unixbench_multi07:55:371
1341599684,53cyclictest24668-21/usr/sbin/munin10:00:290
1341599684,53cyclictest24668-21/usr/sbin/munin10:00:290
13415996834,21cyclictest181rcu_preempt07:55:130
13419996747,15cyclictest27045-21kworker/u17:0-events_unbound10:35:001
13419996746,11cyclictest0-21swapper/110:40:251
13419996746,11cyclictest0-21swapper/110:40:251
13419996745,16cyclictest544-21cron07:34:581
1341999673,23cyclictest0-21swapper/108:37:491
13415996731,12cyclictest181rcu_preempt07:10:190
13415996725,12cyclictest181rcu_preempt08:50:560
13419996645,14cyclictest7353-21sh12:11:511
13419996643,16cyclictest14612-21ntp_offset08:40:261
13419996641,14cyclictest0-21swapper/108:20:321
13419996641,14cyclictest0-21swapper/108:20:321
1341999662,47cyclictest10176-21grep08:25:361
13415996641,11cyclictest181rcu_preempt07:00:310
13415996639,12cyclictest181rcu_preempt09:55:230
1341599663,51cyclictest4902-21/usr/sbin/munin11:15:300
1341999659,15cyclictest0-21swapper/107:47:231
1341999659,15cyclictest0-21swapper/107:47:231
13419996548,11cyclictest297-21systemd-logind08:34:591
1341999652,46cyclictest352-21runrttasks08:48:411
13415996526,9cyclictest0-21swapper/012:12:440
13415996346,12cyclictest58-21kcompactd008:46:360
1341999625,43cyclictest9643-21switchtime12:15:341
1341999622,44cyclictest31126-21switchtime07:50:321
13436996141,13cyclictest451rcuc/308:59:283
1341999613,45cyclictest7479-21latency_hist08:20:001
1341999613,45cyclictest7479-21latency_hist08:20:001
1341999613,44cyclictest26500-21head07:35:351
13419996111,38cyclictest1623-21fschecks_time10:15:211
13419996111,38cyclictest1623-21fschecks_time10:15:211
13436996040,15cyclictest451rcuc/307:45:533
13436996040,15cyclictest451rcuc/307:45:533
1341999603,47cyclictest18962-21grep08:55:141
13436995944,10cyclictest451rcuc/310:32:243
13436995942,10cyclictest451rcuc/309:47:163
13436995938,12cyclictest451rcuc/308:12:463
13436995937,15cyclictest451rcuc/309:02:153
1343699574,37cyclictest11770-21taskset12:20:073
13436995734,17cyclictest451rcuc/307:43:063
1343699573,39cyclictest28906-21taskset11:54:573
130452575,36sleep30-21swapper/306:57:113
1343699562,38cyclictest18137-21taskset11:35:483
1343699553,37cyclictest11207-21taskset11:25:183
13436995442,8cyclictest451rcuc/307:24:183
13436995438,11cyclictest451rcuc/308:31:093
13436995434,15cyclictest451rcuc/310:11:243
1343699543,38cyclictest5843-21chrt10:21:493
1343699543,36cyclictest29910-21chrt10:08:353
1343699543,33cyclictest18813-21taskset07:15:013
1343699542,42cyclictest14724-21chrt09:43:133
13436995341,8cyclictest451rcuc/308:47:253
13436995339,9cyclictest451rcuc/309:39:453
13436995337,9cyclictest451rcuc/311:32:533
13436995337,9cyclictest451rcuc/311:32:533
13436995336,12cyclictest451rcuc/308:03:023
1343699533,35cyclictest29732-21chrt11:01:223
13436995330,16cyclictest451rcuc/307:27:543
13436995240,8cyclictest451rcuc/311:17:493
13436995238,9cyclictest451rcuc/309:22:553
13436995238,10cyclictest451rcuc/307:16:083
13436995236,10cyclictest451rcuc/310:36:083
1343699523,34cyclictest659-21chrt12:00:333
13436995232,13cyclictest451rcuc/310:53:493
1343699522,27cyclictest0-21swapper/309:05:483
1343699514,33cyclictest29946-21chrt09:15:163
13436995139,8cyclictest451rcuc/311:48:423
13436995138,9cyclictest451rcuc/309:57:583
13436995138,9cyclictest451rcuc/309:51:503
13436995138,9cyclictest451rcuc/309:51:503
1343699513,38cyclictest5267-21taskset12:09:283
1343699513,38cyclictest24726-21chrt07:30:583
1343699513,35cyclictest21125-21chrt11:40:463
1343699513,35cyclictest21125-21chrt11:40:463
1343699513,34cyclictest1770-21taskset10:15:243
1343699513,34cyclictest1770-21taskset10:15:243
1343699513,34cyclictest12827-21chrt08:35:253
1343699512,36cyclictest21358-21chrt10:48:123
1343699512,35cyclictest7523-21chrt09:30:363
13436994937,8cyclictest451rcuc/310:01:473
13436994937,8cyclictest451rcuc/310:01:473
13436994936,9cyclictest451rcuc/308:40:573
1343699493,33cyclictest28041-21taskset09:10:463
1343699493,33cyclictest28041-21taskset09:10:463
1343699493,32cyclictest31609-21chrt11:05:193
13436994835,9cyclictest451rcuc/311:13:273
13436994835,9cyclictest451rcuc/311:13:273
13436994835,9cyclictest451rcuc/307:04:583
13436994835,8cyclictest451rcuc/311:57:343
13436994835,8cyclictest451rcuc/311:57:343
1343699483,34cyclictest9536-21taskset11:22:013
1343699483,32cyclictest25383-21chrt10:55:143
1343699483,31cyclictest8514-21chrt08:20:353
1343699483,31cyclictest8514-21chrt08:20:353
13436994833,10cyclictest451rcuc/307:05:393
13436994833,10cyclictest451rcuc/307:05:393
1343699473,33cyclictest8734-21chrt10:26:313
1343699473,33cyclictest8734-21chrt10:26:313
13436994732,10cyclictest451rcuc/307:54:283
1343699472,32cyclictest17428-21chrt12:29:513
1343699472,31cyclictest17429-21chrt08:50:153
1343699472,28cyclictest3755-21chrt08:05:363
1343699464,33cyclictest4864-21taskset09:26:353
107382460,7sleep30-21swapper/312:17:563
13436994531,9cyclictest451rcuc/310:41:123
13436994531,9cyclictest451rcuc/310:41:123
13436994431,9cyclictest451rcuc/308:27:203
13436994431,8cyclictest451rcuc/307:36:503
1343699432,31cyclictest5827-21taskset12:10:143
1343699432,28cyclictest6242-21taskset08:15:203
1343699432,28cyclictest6242-21taskset08:15:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional