You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 05:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Sat Aug 22, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37356021130,3sleep20-21swapper/219:50:172
139099930,75rtkit-daemon0-21swapper/119:05:191
35791928768,13sleep20-21swapper/219:09:112
35782627639,13sleep30-21swapper/319:08:003
35791726739,21sleep00-21swapper/019:09:090
4367522590,5sleep10-21swapper/122:50:201
4217592500,4sleep3421755-21grep22:10:013
3711582500,1sleep30-21swapper/319:45:003
4451302450,2sleep3411rcuc/323:15:013
4451302450,2sleep3411rcuc/323:15:013
4671062420,3sleep00-21swapper/000:15:160
4114192420,3sleep10-21swapper/121:40:111
358112992616,7cyclictest436-20systemd-journal20:00:002
358107992514,4cyclictest13-21ksoftirqd/021:00:010
3661082240,5chrt0-21swapper/319:30:133
35811499248,4cyclictest42-21ksoftirqd/322:00:323
35810799247,5cyclictest13-21ksoftirqd/000:25:010
35810799247,5cyclictest13-21ksoftirqd/000:25:010
358107992411,4cyclictest13-21ksoftirqd/020:31:380
358107992411,4cyclictest13-21ksoftirqd/019:59:420
358107992410,5cyclictest13-21ksoftirqd/020:17:110
35811499239,4cyclictest42-21ksoftirqd/321:40:363
35811499238,13cyclictest374397-21latency_hist19:55:003
358114992318,4cyclictest42-21ksoftirqd/300:10:023
358114992316,5cyclictest372676-21munin-run19:50:003
358112992321,1cyclictest0-21swapper/223:46:432
35810799239,5cyclictest141rcu_preempt20:10:200
35810799239,4cyclictest13-21ksoftirqd/000:31:450
35810799238,4cyclictest13-21ksoftirqd/019:10:530
358107992314,6cyclictest436-20systemd-journal23:09:560
358107992311,4cyclictest13-21ksoftirqd/020:38:550
358107992311,4cyclictest13-21ksoftirqd/020:05:590
35810799231,11cyclictest13-21ksoftirqd/020:30:010
358107992310,4cyclictest13-21ksoftirqd/023:30:100
358107992310,4cyclictest13-21ksoftirqd/023:22:070
358107992310,4cyclictest13-21ksoftirqd/022:19:530
358107992310,4cyclictest13-21ksoftirqd/021:18:260
358107992310,4cyclictest13-21ksoftirqd/021:12:040
358107992310,4cyclictest13-21ksoftirqd/020:48:060
358107992310,4cyclictest13-21ksoftirqd/020:24:080
358107992310,4cyclictest13-21ksoftirqd/020:04:510
358107992310,4cyclictest13-21ksoftirqd/019:45:270
358107992310,4cyclictest13-21ksoftirqd/019:44:140
35811499222,3cyclictest141rcu_preempt22:49:133
358114992220,1cyclictest3056-21PK-Backend21:37:503
358114992220,1cyclictest0-21swapper/322:23:133
358114992220,1cyclictest0-21swapper/321:13:233
358114992220,1cyclictest0-21swapper/320:30:243
358114992218,3cyclictest42-21ksoftirqd/322:34:593
358114992214,6cyclictest883-21dbus-daemon00:14:593
358114992212,4cyclictest42-21ksoftirqd/319:24:223
358112992220,1cyclictest0-21swapper/221:16:302
358112992220,1cyclictest0-21swapper/220:38:172
358112992220,1cyclictest0-21swapper/220:34:522
358112992220,1cyclictest0-21swapper/219:47:262
358112992220,1cyclictest0-21swapper/219:23:402
358112992215,5cyclictest446830-21systemd-run23:20:002
35811299220,21cyclictest0-21swapper/221:08:112
35811299220,20cyclictest0-21swapper/222:43:012
358109992220,1cyclictest3056-21PK-Backend22:12:571
358109992220,1cyclictest0-21swapper/123:48:011
358109992220,1cyclictest0-21swapper/121:45:531
358109992219,2cyclictest3056-21PK-Backend23:51:261
358109992219,2cyclictest3056-21PK-Backend22:06:291
358109992219,2cyclictest3056-21PK-Backend22:02:091
358109992219,2cyclictest3056-21PK-Backend21:21:201
358109992219,2cyclictest3056-21PK-Backend20:52:191
358109992219,2cyclictest3056-21PK-Backend19:30:271
358109992214,6cyclictest473083-21basename00:35:001
35810999220,20cyclictest3056-21PK-Backend23:20:031
35810799229,4cyclictest13-21ksoftirqd/023:53:230
35810799229,4cyclictest13-21ksoftirqd/023:37:560
35810799229,4cyclictest13-21ksoftirqd/022:07:340
35810799229,4cyclictest13-21ksoftirqd/019:51:160
35810799229,4cyclictest13-21ksoftirqd/019:29:470
35810799229,4cyclictest13-21ksoftirqd/019:21:010
35810799229,4cyclictest13-21ksoftirqd/019:16:030
35810799229,4cyclictest13-21ksoftirqd/000:11:120
35810799228,5cyclictest13-21ksoftirqd/022:34:010
35810799225,4cyclictest13-21ksoftirqd/000:37:350
358107992220,1cyclictest0-21swapper/022:53:360
358107992220,1cyclictest0-21swapper/022:39:570
358107992220,1cyclictest0-21swapper/019:36:100
35810799221,4cyclictest13-21ksoftirqd/021:20:110
358107992211,4cyclictest13-21ksoftirqd/020:42:290
358107992211,4cyclictest13-21ksoftirqd/020:42:290
358107992210,4cyclictest13-21ksoftirqd/020:50:430
358107992210,3cyclictest13-21ksoftirqd/021:56:220
4122842210,5sleep23056-21PK-Backend21:40:192
35811499219,4cyclictest42-21ksoftirqd/321:23:053
35811499213,3cyclictest141rcu_preempt00:21:083
35811499213,3cyclictest141rcu_preempt00:21:083
358114992119,1cyclictest3056-21PK-Backend23:01:453
358114992119,1cyclictest3056-21PK-Backend22:59:213
358114992119,1cyclictest3056-21PK-Backend22:51:523
358114992119,1cyclictest3056-21PK-Backend00:15:263
358114992119,1cyclictest0-21swapper/323:36:403
358114992119,1cyclictest0-21swapper/323:33:163
358114992119,1cyclictest0-21swapper/322:19:283
358114992119,1cyclictest0-21swapper/320:55:533
358114992119,1cyclictest0-21swapper/320:01:523
358114992119,1cyclictest0-21swapper/319:25:423
358114992119,1cyclictest0-21swapper/300:31:423
358114992117,3cyclictest42-21ksoftirqd/321:10:013
358114992115,4cyclictest414938-21cpuspeed_turbos21:50:133
35811499211,18cyclictest427589-21df_inode22:25:123
35811499210,2cyclictest447735-21strings23:20:183
35811499210,20cyclictest3056-21PK-Backend23:08:553
35811499210,20cyclictest3056-21PK-Backend22:36:513
35811499210,20cyclictest0-21swapper/320:48:503
35811499210,20cyclictest0-21swapper/320:36:593
35811499210,20cyclictest0-21swapper/319:59:153
358112992119,1cyclictest0-21swapper/223:51:572
358112992119,1cyclictest0-21swapper/223:08:162
358112992119,1cyclictest0-21swapper/223:00:532
358112992119,1cyclictest0-21swapper/222:04:422
358112992119,1cyclictest0-21swapper/221:11:562
358112992119,1cyclictest0-21swapper/220:40:542
358112992119,1cyclictest0-21swapper/220:40:532
358112992119,1cyclictest0-21swapper/220:28:172
358112992119,1cyclictest0-21swapper/219:39:022
358112992119,1cyclictest0-21swapper/219:33:142
358112992119,1cyclictest0-21swapper/200:25:092
358112992116,4cyclictest462463-21latency_hist00:05:002
358112992114,5cyclictest463810-21sshd00:05:552
358112992113,6cyclictest34-21ksoftirqd/222:35:112
35811299210,20cyclictest0-21swapper/220:52:572
35811299210,20cyclictest0-21swapper/220:15:042
35811299210,20cyclictest0-21swapper/200:30:232
35811299210,1cyclictest0-21swapper/222:20:172
35811299210,19cyclictest0-21swapper/220:05:052
358109992119,1cyclictest3056-21PK-Backend23:28:011
358109992119,1cyclictest3056-21PK-Backend22:29:061
358109992119,1cyclictest3056-21PK-Backend22:00:001
358109992119,1cyclictest3056-21PK-Backend21:54:181
358109992119,1cyclictest3056-21PK-Backend00:04:041
358109992119,1cyclictest0-21swapper/121:37:291
358109992118,2cyclictest3056-21PK-Backend21:12:281
358109992118,2cyclictest3056-21PK-Backend19:44:131
358109992115,5cyclictest407643-21systemctl21:30:001
35810999210,20cyclictest3056-21PK-Backend23:37:471
35810999210,20cyclictest3056-21PK-Backend23:32:471
35810999210,20cyclictest3056-21PK-Backend20:18:341
35810999210,20cyclictest3056-21PK-Backend19:26:281
35810999210,20cyclictest3056-21PK-Backend00:08:151
35810999210,20cyclictest0-21swapper/120:10:221
35810999210,19cyclictest3056-21PK-Backend22:32:041
35810999210,19cyclictest3056-21PK-Backend19:15:271
35810799219,4cyclictest13-21ksoftirqd/023:56:010
35810799219,4cyclictest13-21ksoftirqd/022:25:370
35810799219,4cyclictest13-21ksoftirqd/019:33:400
35810799218,4cyclictest13-21ksoftirqd/023:15:270
35810799218,4cyclictest13-21ksoftirqd/021:07:510
35810799216,8cyclictest13-21ksoftirqd/023:14:170
35810799216,8cyclictest13-21ksoftirqd/023:14:170
358107992119,1cyclictest0-21swapper/023:00:320
358107992119,1cyclictest0-21swapper/022:49:030
358107992119,1cyclictest0-21swapper/022:11:580
358107992119,1cyclictest0-21swapper/022:01:100
358107992119,1cyclictest0-21swapper/021:53:190
358107992119,1cyclictest0-21swapper/021:26:150
358107992115,5cyclictest0-21swapper/023:48:370
35810799211,19cyclictest0-21swapper/000:01:150
358107992110,4cyclictest13-21ksoftirqd/023:25:320
358107992110,4cyclictest13-21ksoftirqd/000:08:360
35811499208,4cyclictest42-21ksoftirqd/320:16:113
35811499207,11cyclictest460701-40.mlocate00:00:013
35811499202,16cyclictest445732-21hddtemp_smartct23:15:163
358114992018,1cyclictest3056-21PK-Backend23:45:193
358114992018,1cyclictest3056-21PK-Backend21:32:223
358114992018,1cyclictest0-21swapper/322:13:333
358114992018,1cyclictest0-21swapper/321:45:273
358114992018,1cyclictest0-21swapper/320:50:393
35811499201,6cyclictest361008-21interrupts19:15:173
35811499200,8cyclictest42-21ksoftirqd/320:13:033
35811499200,5cyclictest3056-21PK-Backend00:25:093
35811499200,19cyclictest0-21swapper/321:17:093
35811499200,19cyclictest0-21swapper/320:23:493
35811499200,19cyclictest0-21swapper/319:13:053
358112992018,1cyclictest3056-21PK-Backend22:45:252
358112992018,1cyclictest0-21swapper/223:57:092
358112992018,1cyclictest0-21swapper/223:33:372
358112992018,1cyclictest0-21swapper/221:45:162
358112992018,1cyclictest0-21swapper/220:10:052
35811299200,1cyclictest0-21swapper/221:55:162
35811299200,19cyclictest0-21swapper/222:56:542
35811299200,19cyclictest0-21swapper/221:50:572
35811299200,19cyclictest0-21swapper/221:21:022
35811299200,19cyclictest0-21swapper/219:43:152
35811299200,19cyclictest0-21swapper/219:16:592
35811299200,19cyclictest0-21swapper/200:35:372
35811299200,19cyclictest0-21swapper/200:20:422
35811299200,19cyclictest0-21swapper/200:20:422
358109992018,1cyclictest3056-21PK-Backend22:21:021
358109992018,1cyclictest3056-21PK-Backend21:05:111
358109992018,1cyclictest3056-21PK-Backend20:48:551
358109992018,1cyclictest3056-21PK-Backend20:34:421
358109992018,1cyclictest3056-21PK-Backend19:24:531
358109992018,1cyclictest0-21swapper/122:56:581
358109992018,1cyclictest0-21swapper/122:47:511
358109992018,1cyclictest0-21swapper/121:31:131
358109992017,2cyclictest3056-21PK-Backend20:44:431
358109992017,2cyclictest3056-21PK-Backend20:44:431
35810999200,1cyclictest3056-21PK-Backend21:00:521
35810999200,19cyclictest0-21swapper/123:40:231
35810999200,19cyclictest0-21swapper/100:26:201
35810999200,18cyclictest3056-21PK-Backend19:14:391
35810799209,4cyclictest13-21ksoftirqd/020:56:510
35810799207,4cyclictest13-21ksoftirqd/022:21:400
358107992018,1cyclictest0-21swapper/023:41:490
358107992018,1cyclictest0-21swapper/022:55:100
358107992018,1cyclictest0-21swapper/021:41:230
358107992015,4cyclictest13-21ksoftirqd/022:45:000
358107992012,6cyclictest469873-21apt00:25:110
35810799200,19cyclictest0-21swapper/021:46:160
35811499198,4cyclictest42-21ksoftirqd/323:51:523
35811499198,4cyclictest42-21ksoftirqd/320:07:043
358114991917,1cyclictest0-21swapper/300:01:353
358114991914,3cyclictest474199-21python300:35:203
358114991911,6cyclictest42-21ksoftirqd/323:41:473
358114991911,6cyclictest42-21ksoftirqd/321:26:173
358114991910,4cyclictest42-21ksoftirqd/321:03:593
35811499190,6cyclictest368326-21expr19:35:163
35811499190,1cyclictest3056-21PK-Backend22:40:373
35811499190,1cyclictest0-21swapper/321:56:023
35811499190,1cyclictest0-21swapper/320:42:003
35811499190,1cyclictest0-21swapper/320:42:003
358112991917,1cyclictest0-21swapper/223:23:512
358112991917,1cyclictest0-21swapper/222:50:462
358112991917,1cyclictest0-21swapper/221:27:592
358112991917,1cyclictest0-21swapper/220:46:562
358112991917,1cyclictest0-21swapper/219:10:302
35811299190,1cyclictest0-21swapper/223:42:172
35811299190,1cyclictest0-21swapper/221:00:072
35811299190,1cyclictest0-21swapper/220:59:192
35811299190,1cyclictest0-21swapper/200:11:512
35811299190,18cyclictest3056-21PK-Backend22:27:482
35811299190,18cyclictest0-21swapper/222:32:202
35811299190,18cyclictest0-21swapper/222:16:322
35811299190,18cyclictest0-21swapper/200:16:172
358109991917,1cyclictest3056-21PK-Backend23:59:511
358109991917,1cyclictest3056-21PK-Backend21:15:241
358109991917,1cyclictest3056-21PK-Backend20:58:421
358109991917,1cyclictest3056-21PK-Backend19:55:341
358109991917,1cyclictest3056-21PK-Backend00:11:411
358109991917,1cyclictest0-21swapper/122:38:461
358109991913,4cyclictest446801-21ls23:19:591
35810999190,1cyclictest0-21swapper/122:42:311
35810999190,1cyclictest0-21swapper/100:15:051
35810799190,18cyclictest0-21swapper/021:37:520
35811499180,16cyclictest449125-21expr23:25:113
35811299182,12cyclictest364008-21apt19:25:092
358112991816,1cyclictest3056-21PK-Backend21:30:222
358112991816,1cyclictest0-21swapper/223:39:392
358112991816,1cyclictest0-21swapper/222:13:352
358112991812,4cyclictest445059-21cut23:15:012
358112991812,4cyclictest445059-21cut23:15:012
358109991815,2cyclictest3056-21PK-Backend22:18:041
358112991712,3cyclictest384119-21open_inodes20:20:202
358112991710,5cyclictest376950-21ls20:00:172
358109991712,4cyclictest468517-21diskstats00:20:141
358109991712,4cyclictest468517-21diskstats00:20:131
358109991712,4cyclictest441753-21expr23:05:131
358109991712,3cyclictest444425-21vmstat23:10:221
358109991712,3cyclictest444425-21vmstat23:10:221
358109991712,3cyclictest441340-21latency_hist23:05:011
358109991711,5cyclictest379423-21sshd20:07:561
358107991712,4cyclictest408352-21interrupts21:30:200
358114991610,5cyclictest385438-21expr20:25:123
35811299169,6cyclictest421293-21sed22:05:222
35811299161,13cyclictest450297-20unin-run23:30:002
358109991613,2cyclictest3056-21PK-Backend20:39:431
35810999157,2cyclictest141rcu_preempt00:40:001
35811299149,4cyclictest886-21NetworkManager21:38:032
358109991410,3cyclictest3056-21PK-Backend19:50:401
35810999139,3cyclictest3056-21PK-Backend20:27:521
35810999139,3cyclictest3056-21PK-Backend20:23:481
35810999139,3cyclictest3056-21PK-Backend19:49:361
35810999139,3cyclictest3056-21PK-Backend19:36:161
35810999138,4cyclictest3056-21PK-Backend20:00:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional