You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 15:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8.osadl.org (updated Tue Nov 18, 2025 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2644367993228,1cyclictest0-21swapper/308:45:013
2644367993128,1cyclictest0-21swapper/312:00:013
2644367992623,1cyclictest0-21swapper/311:45:013
2644367992623,1cyclictest0-21swapper/309:26:543
2644367992522,1cyclictest0-21swapper/307:25:163
2644367992320,1cyclictest0-21swapper/310:40:003
264436799221,15cyclictest2911033-21chrt10:20:003
264436799220,20cyclictest3048768-21turbostat11:49:563
2644367992118,1cyclictest0-21swapper/310:05:263
264436799211,17cyclictest2959763-21sleep10:50:013
2644367991916,1cyclictest0-21swapper/311:35:013
2644367991916,1cyclictest0-21swapper/308:20:213
264436799191,14cyclictest2684139-21turbostat07:40:003
264436799191,13cyclictest2854596-21chrt09:45:013
264436799191,13cyclictest2650572-21turbostat07:15:013
2644367991815,1cyclictest0-21swapper/312:30:213
2644367991815,1cyclictest0-21swapper/312:05:173
2644367991815,1cyclictest0-21swapper/311:50:213
2644367991815,1cyclictest0-21swapper/311:20:183
28737972170,9sleep12875903-21cut09:55:191
264436799177,8cyclictest2728003-21turbostat08:19:553
264436799177,7cyclictest29503562sleep310:42:093
2644367991714,1cyclictest0-21swapper/310:22:433
2644367991714,1cyclictest0-21swapper/308:35:013
2644367991714,1cyclictest0-21swapper/307:15:023
264436799171,13cyclictest3080434-21taskset12:03:393
264436799171,13cyclictest3019453-21taskset11:25:203
264436799171,13cyclictest2884898-21chrt10:00:333
264436799171,12cyclictest3129730-21taskset12:35:053
264436799171,12cyclictest2862414-21sleep09:54:563
264436799171,12cyclictest2836996-21chrt09:32:163
264436799171,12cyclictest2806132-21taskset09:15:153
264436799171,12cyclictest2805571-21turbostat09:15:003
264436799171,12cyclictest2775311-21taskset08:50:203
264436799171,12cyclictest2775311-21taskset08:50:193
264436799170,4cyclictest2671508-21turbostat07:34:563
264436799167,6cyclictest29922452sleep311:10:073
264436799167,6cyclictest27913732sleep309:05:163
2644367991613,1cyclictest0-21swapper/311:35:163
2644367991613,1cyclictest0-21swapper/311:15:123
2644367991613,1cyclictest0-21swapper/310:30:173
264436799161,12cyclictest3105510-21taskset12:20:063
264436799161,12cyclictest3094507-21taskset12:10:203
264436799161,12cyclictest2990555-21taskset11:06:363
264436799161,12cyclictest2968038-21chrt10:55:023
264436799161,12cyclictest2925498-21chrt10:25:333
264436799161,12cyclictest2871025-21taskset09:55:113
264436799161,12cyclictest2727288-21chrt08:10:243
264436799161,12cyclictest2708892-21taskset07:56:103
264436799161,12cyclictest2690027-21taskset07:40:223
264436799161,11cyclictest3119447-21taskset12:25:323
264436799161,11cyclictest2844469-21taskset09:35:433
264436799160,13cyclictest2715523-21turbostat08:04:593
2644367991512,1cyclictest0-21swapper/307:20:173
264436799151,12cyclictest2983379-21chrt11:03:453
264436799151,12cyclictest2901835-21taskset10:12:183
264436799151,12cyclictest2702863-21chrt07:54:133
264436799151,11cyclictest3102560-21taskset12:15:223
264436799151,11cyclictest2960067-21taskset10:50:133
264436799151,11cyclictest2846546-21taskset09:40:053
264436799151,11cyclictest2821032-21chrt09:22:583
264436799151,11cyclictest2790182-21taskset09:00:213
264436799151,11cyclictest2784661-21taskset08:59:383
264436799150,12cyclictest2753560-21chrt08:35:123
264436799150,12cyclictest2721475-21taskset08:06:363
30410852141,10sleep13041052-21/usr/sbin/munin11:40:131
264436799141,11cyclictest2746384-21taskset08:25:273
264436799140,11cyclictest2766096-21taskset08:45:023
264436799140,11cyclictest2696404-21taskset07:46:403
264436799140,11cyclictest2644473-21chrt07:10:003
28458952131,9sleep12845902-21sh09:39:071
26382612138,1sleep20-21swapper/207:05:122
147598130,8rtkit-daemon2892985-21ssh10:05:292
29844972121,8sleep02984702-21hddtemp_smartct11:05:180
29761482121,7sleep02976183-21apt-get11:00:140
29499642121,8sleep22949970-21ssh10:41:112
147598120,8rtkit-daemon2784973-21/usr/sbin/munin09:00:260
147598120,8rtkit-daemon2753063-21apt-get08:35:000
147598120,7rtkit-daemon2638262-21/usr/sbin/munin07:05:211
147598120,7rtkit-daemon1540-21sendmail-mta08:42:240
29602212111,7sleep02960297-21grep10:50:160
28857012110,8sleep12885709-21ssh10:02:231
147598110,7rtkit-daemon2650733-21munin-node09:15:150
31057762100,7sleep03105983-21hddtemp_smartct12:20:170
30959552101,7sleep00-21swapper/012:12:150
30777142100,7sleep03078394-21expr12:00:210
29578372100,7sleep12957843-21ssh10:45:351
28548982100,7sleep12855029-21expr09:45:171
28476072100,7sleep02848725-21cut09:40:180
28226672100,7sleep02823247-21irqrtprio09:25:150
27665132101,6sleep12767070-21irqrtprio08:45:171
27345592100,7sleep12734651-21cpuspeed_turbos08:20:131
26678332101,6sleep12669891-21kthreadcore07:25:191
147598100,7rtkit-daemon515-21gmain08:05:280
147598100,6rtkit-daemon313-21systemd-udevd10:25:431
147598100,6rtkit-daemon313-21systemd-udevd10:09:431
147598100,6rtkit-daemon3121723-21cat12:30:161
147598100,6rtkit-daemon3007134-21sh11:17:430
147598100,4rtkit-daemon2650733-21munin-node09:49:252
147598100,3rtkit-daemon214-21plymouthd10:17:202
147598100,3rtkit-daemon214-21plymouthd08:49:000
3096471290,2sleep20-21swapper/212:13:292
3081367291,4sleep13081404-21apt-get12:05:151
3073295291,4sleep13073332-21apt-get12:00:131
3072927291,5sleep03072962-21apt12:00:010
3062401290,7sleep03062467-21ntp_states11:50:220
3026315290,1sleep13028013-21kthreadcore11:30:151
3024886291,4sleep03024922-21apt-get11:30:100
3010122290,6sleep23012021-21kthreadcore11:20:202
2950602290,7sleep12910156-21kworker/u16:4+xprtiod10:42:401
2943929291,5sleep02944088-21diskstats10:40:180
2942724290,1sleep10-21swapper/110:38:101
2928086290,2sleep12928410-21sed10:30:171
2838592290,6sleep02838752-21df_inode09:35:180
2822217290,1sleep12822225-21ssh09:25:081
2813735290,6sleep22813889-21latency_hist09:20:002
2784972290,6sleep22785005-21apt-get09:00:132
2723229290,6sleep12724364-21cut08:10:191
2658903291,4sleep172-21kcompactd007:19:501
14759890,5rtkit-daemon720-21snmpd11:12:180
14759890,5rtkit-daemon720-21snmpd07:45:130
14759890,5rtkit-daemon2886443-21ssh10:04:042
14759890,5rtkit-daemon2870310-21ssh09:54:031
14759890,5rtkit-daemon0-21swapper/110:12:001
14759890,4rtkit-daemon476-21wpa_supplicant09:43:032
14759890,4rtkit-daemon2833535-21/usr/sbin/munin09:30:190
14759890,3rtkit-daemon2650733-21munin-node11:01:052
14759890,2rtkit-daemon214-21plymouthd08:56:320
14759890,1rtkit-daemon0-21swapper/008:12:300
3112298281,4sleep2720-21snmpd12:22:392
3054559281,4sleep03054804-21/usr/sbin/munin11:45:260
3032914281,3sleep13032949-21apt-get11:35:131
3016833281,4sleep13016868-21apt-get11:25:161
3013703281,4sleep13013765-21ntp_states11:20:221
2999865280,6sleep22999872-21sh11:14:392
2984275281,3sleep12984316-21apt-get11:05:151
2952937283,3sleep02953807-21kthreadcore10:45:180
2869921284,2sleep20-21swapper/209:53:152
2863870281,4sleep02864782-21cut09:50:160
2837594280,2sleep22837602-21ssh09:33:402
2830155280,6sleep10-21swapper/109:30:021
2816132280,5sleep02818119-21kthreadcore09:20:190
2797781281,3sleep02797820-21apt-get09:10:130
2752222280,5sleep12752347-21/usr/sbin/munin08:30:231
2733489280,5sleep22733623-21sed08:15:242
2704586280,6sleep22706210-21kthreadcore07:55:182
2703185281,3sleep12703218-21apt-get07:55:131
2702926280,5sleep12703107-21latency_hist07:55:001
2696954283,3sleep2431ktimers/207:50:122
14759880,4rtkit-daemon720-21snmpd07:10:211
14759880,4rtkit-daemon515-21gmain08:30:010
14759880,4rtkit-daemon3119792-21ssh12:26:231
14759880,4rtkit-daemon2886805-21kworker/u16:2+rpciod10:51:322
14759880,4rtkit-daemon2734709-21df_inode08:20:140
14759880,4rtkit-daemon2684103-21chrt07:39:520
14759880,4rtkit-daemon214-21plymouthd12:17:530
14759880,4rtkit-daemon214-21plymouthd11:36:272
14759880,4rtkit-daemon214-21plymouthd09:09:210
14759880,4rtkit-daemon0-21swapper/008:28:360
14759880,2rtkit-daemon0-21swapper/210:20:082
14759880,1rtkit-daemon214-21plymouthd11:52:081
3129811271,2sleep13129850-21apt-get12:35:161
3128276271,4sleep23128282-21ssh12:32:172
3114135270,5sleep03115080-21irqstats12:25:170
3103799270,5sleep12797450-21diskmemload12:16:481
3073269271,4sleep23072461-21kworker/u16:3+nfsiod12:00:102
3047157271,3sleep23047164-21sh11:41:152
3022024273,2sleep2421rcuc/211:25:242
3006215270,4sleep13006225-21ssh11:15:331
2977583270,5sleep12978792-21grep11:00:191
2973628271,3sleep12973872-21ssh10:55:291
2973385271,3sleep22973394-21ntp_states10:55:262
2973254270,1sleep00-21swapper/010:55:200
2965453270,4sleep12965643-21sed10:50:251
2933993270,2sleep20-21swapper/210:31:462
2909498270,4sleep02909505-21sshd10:16:340
2903794271,3sleep12904740-21irqstats10:15:191
2900814270,1sleep20-21swapper/210:10:262
2812608270,1sleep10-21swapper/109:17:231
2805368271,4sleep12805376-21ssh09:14:391
2788930271,3sleep12790042-21memory09:00:211
2783734271,3sleep12783811-21ntp_states08:55:231
2777578271,2sleep00-21swapper/008:50:300
2777578271,2sleep00-21swapper/008:50:290
2765819270,1sleep20-21swapper/208:44:392
2753039270,4sleep22753363-21sed08:35:012
2678017271,3sleep22678049-21apt-get07:35:142
2671489270,4sleep20-21swapper/207:30:012
2654400273,2sleep22656376-21kthreadcore07:15:212
14759870,4rtkit-daemon730-21ntpd08:18:500
14759870,4rtkit-daemon720-21snmpd07:40:420
14759870,4rtkit-daemon2650733-21munin-node08:00:410
14759870,3rtkit-daemon720-21snmpd12:08:190
14759870,3rtkit-daemon515-21gmain07:56:320
14759870,3rtkit-daemon3054985-21ssh11:45:411
14759870,3rtkit-daemon2935678-21ssh10:35:062
14759870,3rtkit-daemon214-21plymouthd11:30:482
14759870,3rtkit-daemon1-21systemd11:05:062
14759870,3rtkit-daemon1-21systemd09:38:192
14759870,3rtkit-daemon1-21systemd07:31:060
14759870,3rtkit-daemon1-21systemd07:23:040
14759870,2rtkit-daemon214-21plymouthd10:21:351
3135668261,3sleep2214-21plymouthd12:35:532
3129372260,1sleep00-21swapper/012:34:520
3116471260,3sleep23118554-21cut12:25:192
3091906260,3sleep13093926-21tr12:10:201
3082793261,3sleep23082830-21kthreadcore12:05:202
3065225261,2sleep10-21swapper/111:55:131
3065199260,1sleep20-21swapper/211:55:102
3056828260,3sleep23056878-21munin-run11:50:012
3040977260,1sleep00-21swapper/011:40:040
3021546260,3sleep03021879-21memory11:25:230
3008467260,3sleep03008504-21apt11:20:130
3006668260,1sleep20-21swapper/211:16:382
2957341261,2sleep20-21swapper/210:45:242
2911611260,3sleep02911773-21diskstats10:20:140
2887216260,4sleep02887374-21df_inode10:05:160
2884873260,1sleep00-21swapper/010:00:300
2878145260,1sleep20-21swapper/209:58:362
2854736260,3sleep02854897-21/usr/sbin/munin09:45:160
2848727260,4sleep12850751-21tr09:40:191
2821082261,3sleep12821090-21ssh09:23:051
2821023261,3sleep2214-21plymouthd09:22:582
2797245260,1sleep10-21swapper/109:07:331
2773751261,2sleep22775309-21cut08:50:202
2773751261,2sleep22775309-21cut08:50:192
2771925260,1sleep20-21swapper/208:47:542
2765610260,1sleep10-21swapper/108:41:571
2755826261,3sleep12757925-21tr08:35:191
2746028260,3sleep12740748-21/usr/sbin/munin08:25:251
2736796261,3sleep22738876-21kthreadcore08:20:182
2727644260,1sleep20-21swapper/208:10:442
2721739260,3sleep22721761-21munin-run08:10:002
2721663260,1sleep10-21swapper/108:09:021
2715311260,1sleep10-21swapper/108:02:371
2709938260,3sleep22710870-21irqstats08:00:182
2702683260,1sleep00-21swapper/007:51:540
2692975261,3sleep12695026-21kthreadcore07:45:191
2684796261,3sleep244-21ksoftirqd/207:40:142
2677636260,1sleep10-21swapper/107:33:321
2671783260,3sleep22671929-21df07:30:122
2658791260,1sleep00-21swapper/007:18:250
2650308260,1sleep20-21swapper/207:11:542
14759860,3rtkit-daemon214-21plymouthd07:25:040
3136641250,2sleep03136649-21ssh12:38:030
3105529251,2sleep10-21swapper/112:20:101
3103560250,3sleep20-21swapper/212:16:182
3063712250,3sleep20-21swapper/211:52:172
3040687250,3sleep03040712-21latency_hist11:40:010
2992292250,2sleep10-21swapper/111:10:131
2942062250,3sleep00-21swapper/010:36:370
2921808250,2sleep00-21swapper/010:25:180
2920104250,3sleep22920662-21ps10:25:172
2895447250,3sleep0214-21plymouthd10:10:160
2829400251,2sleep22829408-21ssh09:28:262
2799648250,3sleep22801569-21kthreadcore09:10:192
2772350250,2sleep12772383-21apt-get08:50:151
2772350250,2sleep12772383-21apt-get08:50:141
2759351250,3sleep20-21swapper/208:37:162
2729674250,2sleep10-21swapper/108:15:191
2683701250,2sleep12683759-21file07:36:011
2927877240,2sleep02928085-21hddtemp_smartct10:30:160
2876242240,1sleep00-21swapper/009:55:220
2797258240,2sleep2214-21plymouthd09:07:422
2780041240,2sleep22781679-21ps08:55:212
2745718240,2sleep22740748-21/usr/sbin/munin08:25:222
2690374241,1sleep10-21swapper/107:44:421
2664920240,3sleep20-21swapper/207:21:082
2662504240,2sleep1214-21plymouthd07:20:171
2644723240,2sleep02644923-21smartctl07:10:190
2644017240,1sleep00-21swapper/007:07:350
2690647230,1sleep20-21swapper/207:45:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional